From bwaumg@urc.tue.nl Thu Oct 1 01:12:57 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Thu, 01 Oct 1998 02:12:57 +0200 Subject: [XML-SIG] checked and response attributes Message-ID: <199810010012.CAA02219@asterix.urc.tue.nl> > Marc van Grootel writes: > > I suggested them in the very beginning because I thought they > > would be useful (not because it is tracked by MSIE - maybe it does > > but, does it?). > > I think I sent my message before finishing my response on this. > As far as I can tell, it does not. But that doesn't mean MSIE > doesn't stash the information somewhere; I could only find the > individual files that represent each bookmark. The only information > was the title (encoded as the file name, sheesh!) yep all '/' ':' and probably others are translated to '_' so you lose information. Maybe for MSIE translations we can just add data (such as the 'real' title) tot the url shortcut file, it's an .ini file and I tried adding something which didn't cause any problems. >, the URL, and the > modification time (in some undetermined format). anyone crack that yet? > We might want to drop the attributes from > and create a profile for it. fine with me. You use the term profile. From what I read about Web collections it seems this is a very loose term. Looking for an XML way of defining a profile I would say Architectural Forms are an other (standardized) answer to the same question. But even then the validation of data values is very limited in SGML and very very limited in XML. What about the date/time attributes: follow the w3c NOTE about Date and Time Formats? (ex. 1998-09-29T18:30:34-05:00) Marc --- Marc van Grootel bwaumg@urc.tue.nl From jeremy@allaire.com Thu Oct 1 04:56:54 1998 From: jeremy@allaire.com (Jeremy Allaire) Date: Wed, 30 Sep 1998 23:56:54 -0400 Subject: [XML-SIG] Python and WDDX Message-ID: <001501bdecef$887c0550$33c3f3ce@jallaire_lt.allaire.com> This is a multi-part message in MIME format. ------=_NextPart_000_0012_01BDECCE.004FCD90 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello All- This is an inquiry and call for interest in helping build Python modules = for WDDX. WDDX (Web Distributed Data Exchange) is an Allaire invented = XML-based technology for exchanging complex data between Web programming = languages. It is implemented as an XML 1.0 DTD and a set of = Serializers/Deserizers for each language platform. Allaire is making = the technology available for free, as open source, and is actively = working with other language communities to implement WDDX interfaces. = For general background information: http://www.allaire.com/developer/wddx/ Allaire and various third-parties have already created modules for = JavaScript 1.x, ColdFusion, Perl, COM/ASP and are working on the Java = implementation. To really make the benefits of something like WDDX = complete for everyone, we're really hoping to see Python and PHP = implementations as well. At a high-level, WDDX provides a lowest-common denominator approach to = distributed web applications based on the really existing languages that = are driving the Web today. =20 First, we'd love general feedback: is this a good/useful thing, would = Python developers benefit from an implementation? Second, what specific questions do you have about the implementation? = About the various language implementations (e.g. what's involved, etc.)? Third, we are actively seeking developers who want to build the Python = modules. There's no money in this for Allaire or anyone else; WDDX and = a forthcoming WDDX SDK will be a freeware, open source project. Please feel to reply to all (e.g. the list), or direct questions = directly to myself, Nate Zelnick or Simeon Simeonev, the architect of = WDDX. Kind regards, Jeremy Allaire ------=_NextPart_000_0012_01BDECCE.004FCD90 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hello All-

This is an inquiry and call for = interest in=20 helping build Python modules for WDDX. WDDX (Web Distributed Data=20 Exchange) is an Allaire invented XML-based technology for exchanging = complex=20 data between Web programming languages. It is implemented as an = XML 1.0=20 DTD and a set of Serializers/Deserizers for each language = platform. =20 Allaire is making the technology available for free, as open source, and = is=20 actively working with other language communities to implement WDDX=20 interfaces. For general background information:

&nbs= p; http://www.allaire.com/de= veloper/wddx/

Allaire and various third-parties = have already=20 created modules for JavaScript 1.x, ColdFusion, Perl, COM/ASP and are = working on=20 the Java implementation. To really make the benefits of something = like=20 WDDX complete for everyone, we're really hoping to see Python and PHP=20 implementations as well.

At a high-level, WDDX provides a = lowest-common=20 denominator approach to distributed web applications based on the really = existing languages that are driving the Web today.

First, we'd love general = feedback: is this=20 a good/useful thing, would Python developers benefit from an=20 implementation?

Second, what specific questions do = you have=20 about the implementation? About the various language = implementations (e.g.=20 what's involved, etc.)?

Third, we are actively seeking = developers who=20 want to build the Python modules. There's no money in this for = Allaire or=20 anyone else; WDDX and a forthcoming WDDX SDK will be a freeware, open = source=20 project.

Please feel to reply to = all (e.g. the=20 list), or direct questions directly to myself, Nate Zelnick or Simeon = Simeonev,=20 the architect of WDDX.

Kind regards,

Jeremy = Allaire

------=_NextPart_000_0012_01BDECCE.004FCD90-- From akuchlin@cnri.reston.va.us Thu Oct 1 15:04:36 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Thu, 1 Oct 1998 10:04:36 -0400 (EDT) Subject: [XML-SIG] Python and WDDX In-Reply-To: <001501bdecef$887c0550$33c3f3ce@jallaire_lt.allaire.com> References: <001501bdecef$887c0550$33c3f3ce@jallaire_lt.allaire.com> Message-ID: <13843.35215.875624.420401@amarok.cnri.reston.va.us> Jeremy Allaire writes: >For general background information: > http://www.allaire.com/developer/wddx/ Note that the links for the chapter on WDDX and the link for the DTD are broken. The DTD can be found at http://www.codebits.com/wddx/wddx_0090.dtd.txt . Now... We need to come to some decision about a course of action here. There are a bunch of proposals for this sort of data -> XML marshalling; WWDX, Lotos, and related things like XML-RPC. I'm not aware that any of these proposals are on the path toward becoming an RFC, or even a W3C recommendation. (Incidentally, the status of the DOM PR should be announced today.) So, there's no clear best candidate for implementation. Question: Should some such protocol be part of the basic package? If so, which one(s)? All of them? My answers to the above questions are "yes", "I have no idea", and "Maybe." -- A.M. Kuchling http://starship.skyport.net/crew/amk/ It has also been referred to as the "Don Beaudry *hack*," but that's a misnomer. There's nothing hackish about it -- in fact, it is rather elegant and deep, even though there's something dark to it. -- GvR, _Metaclass Programming in Python 1.5_ From Fred L. Drake, Jr." checked and response attributes In-Reply-To: <199810010012.CAA02219@asterix.urc.tue.nl> References: <199810010012.CAA02219@asterix.urc.tue.nl> Message-ID: <13843.41285.800712.119608@weyr.cnri.reston.va.us> Marc van Grootel writes: > You use the term profile. From what I read about Web collections it > seems this is a very loose term. Looking for an XML way of defining a > profile I would say Architectural Forms are an other (standardized) I'm not familliar with "Web collections", other than having heard the term. A profile, in the sense I'm using it, is simply a description of how a standard is applied to an application. This can be done ease processing complexity by reducing the range of valid input. In the case of date formats, the profile you cite says that only particular formats defined by a standard should be used for date/time values on the Web, eliminating any requirement for software to also support a range of obscure date formats that don't meaningfully apply. In particular, ISO 8601 also supports week-based dates, partial dates (unspecified year, month), and periods. > What about the date/time attributes: follow the w3c NOTE about Date > and Time Formats? (ex. 1998-09-29T18:30:34-05:00) A profile of ISO 8601! Yes, this is what I'm using in the Grail implementation of XBEL. The xml.utils.iso8601 module supports this profile quite reasonably. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From Fred L. Drake, Jr." --rUOjWGCBJo Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit I've attached what I now have as the XBEL DTD. Since the last post, I've removed the "id" and "lang" attributes from . "id" was removed because it was the only thing that had an ID that should not be referred to by an alias. "lang" was removed because XML supports the "xml:lang" attribute for this purpose. Note the xml:lang still has to be declared to be valid, but I expect use of it for this application will be limited at best, so I've left it out for now. In addition, I've added two new parameter entities, common.attrs and node.attrs. common.attrs is included in the attribute list for all elements other than , and is defined to be empty. node.attrs is only used on the and elements, since those are the essential location for attributes relating to the nodes of the bookmark tree ( still uses to carry the attributes, so this is not needed on ). I used this to defined the "id" and "added" attributes since those are shared. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 --rUOjWGCBJo Content-Type: text/xml Content-Description: Proposed XBEL DTD Content-Disposition: inline; filename="xbel.dtd" Content-Transfer-Encoding: 7bit --rUOjWGCBJo-- From john@totten.com Fri Oct 2 08:00:58 1998 From: john@totten.com (John Totten) Date: Thu, 01 Oct 1998 23:00:58 -0800 Subject: [XML-SIG] MetaKit Message-ID: <36147A2A.B7040B1@totten.com> This should be of interest to you all. The Python port is new. http://www.equi4.com/metakit/summary.html Make sure you look at the performance of http://www.equi4.com/metakit/xml/ Its a lightening fast way of using indexed access to a tree view such as that held in an xml document architecture. The example xml file is the Bible. John Totten From fleck@informatik.uni-bonn.de Fri Oct 2 14:54:18 1998 From: fleck@informatik.uni-bonn.de (Markus Fleck) Date: Fri, 2 Oct 1998 15:54:18 +0200 (MET DST) Subject: [XML-SIG] (fwd) VoxML Message-ID: <199810021354.PAA17331@sokrates.informatik.uni-bonn.de> FYI. -- forwarded message -- Newsgroups: comp.lang.python From: "Steven D. Majewski" Subject: OFF TOPIC: VoxML Message-ID: Sender: daemon@cwi.nl (0000-Admin(0000)) Date: Fri, 2 Oct 1998 13:31:09 GMT Entirely off topic, except perhaps as a development topic -- check out http://VoxML.mot.com/ Motorola's XML based voice mark-up language. ---| Steven D. Majewski (804-982-0831) |--- ---| Department of Molecular Physiology and Biological Physics |--- ---| University of Virginia Health Sciences Center |--- ---| P.O. Box 10011 Charlottesville, VA 22906-0011 |--- "I'm not as big a fool as I used to be, I'm a smaller fool." - Jack Kerouac Some of the Dharma -- end of forwarded message -- From fleck@informatik.uni-bonn.de Fri Oct 2 15:00:39 1998 From: fleck@informatik.uni-bonn.de (Markus Fleck) Date: Fri, 2 Oct 1998 16:00:39 +0200 (MET DST) Subject: [XML-SIG] (fwd) WWW: LUTE Project has a website Message-ID: <199810021400.QAA17366@sokrates.informatik.uni-bonn.de> -- forwarded message -- Path: news.rhrz.uni-bonn.de!RRZ.Uni-Koeln.DE!news-koe1.dfn.de!main.de.uu.net!nntp.newengland.verio.net!news.pn.com!newshub.northeast.verio.net!howland.erols.net!newsfeed1.telenordia.se!newsfeed1.funet.fi!news.helsinki.fi!not-for-mail From: Robin Miller Newsgroups: comp.os.linux.announce Subject: WWW: LUTE Project has a website Followup-To: comp.os.linux.misc Date: Fri, 2 Oct 1998 13:31:07 GMT Organization: Frontier GlobalCenter Inc. Lines: 38 Approved: linux-announce@news.ornl.gov (Mikko Rauhala) Message-ID: Reply-To: roblimo@primenet.com NNTP-Posting-Host: laulujoutsen.pc.helsinki.fi Old-Date: Tue, 29 Sep 1998 08:10:31 -0400 X-No-Archive: yes X-Auth: PGPMoose V1.1 PGP comp.os.linux.announce iQCVAgUBNhTVr1rUI/eHXJZ5AQHkJgP+NnXvrKjj5v1mj8GeQjw6VXjpljbu0nJr QQXEtP/o1Qn4Rm3uf1sbavoC3MU/qatnzJr7Na8LLSwULtPjFoQT8ItuAMs/zJJ8 y83s7SIxSIOV2CicLZqLVOffzybsDS5T/T5R/LuuJ5IzPjQtEp+r0zQ1D+nVGFDU Mhf8sTGxxEk= =88Db -----BEGIN PGP SIGNED MESSAGE----- The Linux Usability Testing and Evaluation [LUTE] project finally has a website @ . Please check it out, sign up for the mailing list, and help this embroyonic effort succeed. It is the brainchild of Jeremy Arnold, who deserves many kudos for having had the original idea, and now, for following through with it. - -- Robin Miller Editor and Mouthpiece http://www.techsightings.com Cheap Computing columnist http://www.andovernews.com - -- This article has been digitally signed by the moderator, using PGP. http://www.iki.fi/mjr/cola-public-key.asc has PGP key for validating signature. Send submissions for comp.os.linux.announce to: linux-announce@news.ornl.gov PLEASE remember a short description of the software and the LOCATION. This group is archived at http://www.iki.fi/mjr/linux/cola.html -----BEGIN PGP SIGNATURE----- Version: 2.6.3i Charset: latin1 iQCVAgUBNhTVpVrUI/eHXJZ5AQE2FgP9HiyrwgS5vBBhd8aQm//03JhAaInXauu0 qIBbR/fgb0IKk0DUj5+23AQc5/BEwx/nIIGY/FHA45YXOv4pe7AIOMT9IJUtga/H B6ZQ0SXS4SySC4D8JYf0Pn8jiTbt6VnUPYhwbO1OlgeVj2ihAZ8SxUZY78/I5P2B eTqyzbzVh1E= =Rh0i -----END PGP SIGNATURE----- -- end of forwarded message -- -- /////////////////////////////////////////////////////////////////////////// Markus B Fleck - Student - University of Bonn - CS Dept IV - UNIX support "May you live in interesting times." (ancient Chinese curse) \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ From fleck@xmailer.informatik.uni-bonn.de Fri Oct 2 15:02:18 1998 From: fleck@xmailer.informatik.uni-bonn.de (Markus Fleck) Date: Fri, 2 Oct 1998 16:02:18 +0200 (MET DST) Subject: [XML-SIG] Sorry Message-ID: <199810021402.QAA17390@sokrates.informatik.uni-bonn.de> Sorry, the last forward was not for xml-sig. I was a bit too quick there accepting the e-mail address for forwarding. Sorry, Markus. -- //////////////////////////////////////////////////////////////////////////// Markus B Fleck - University of Bonn - CS Department IV - WHOIS MF5079 UNIX Administrator - comp.lang.python.announce Moderator "GNU Gather" Free Internet Groupware Project - http://cscw.net/gather/ \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ From bwaumg@urc.tue.nl Fri Oct 2 16:05:56 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Fri, 02 Oct 1998 17:05:56 +0200 Subject: [XML-SIG] Proposed XBEL DTD Message-ID: <199810021505.RAA17786@asterix.urc.tue.nl> Just before the weekend I've got a few possible issues for the XBEL DTD. I say possible because it's not my intention to upset the current design. After reading some stuff about metadata I can see that it's a big and complicated thing and could easily lead to a complicated DTD which should be avoided for XBEL. Anyway ... HTML entities A while ago I suggested adding the ISO Latin 1 entities (like HTML does) was that ruled out? It would keep XBEL more readable. Folders The name for the folder element is directly derived from common bookmark files. In some ways the 'folder' is like the 'sect' in the DocBook DTD. One interpretation of a folder is 'a grouped set of nodes' the fact that this is rendered as a real folder in a bookmark file is a presentational aspect. I see a real advantage when I can use a folder element as just a group of nodes inside the current folder without always being rendered as a separate folder. Such a group-of-nodes is then used only for scoping an info block. To be able to support this we need a distinguishing feature (attribute on folder?) to express what type of folder is meant. When there is no such feature we don't have a choice (other than, for example, the processor that says 'I render all folders deeper then 10 levels in the hierarchy in-line'). It's a subtle issue, it means treating folder as a structuring element but not per se as a presentational structuring element. So I suggest adding an attribute to folder like: type (group|folder) 'folder' url: problem 1 In the current DTD 'alias' can be a 'link' to an url or a folder. I think this is good (MSIE favorites supports it with shortcuts). However there's an asymmetry: an 'url' alias does not point to an element with info, a 'folder' alias does. So resolving the 'url' alias (including the info) is now different from resolving a 'folder' alias. This is a result from our decision to accept 'bare' url's. This moved the id attribute from bookmark to the url element. url: problem 2 Now both 'bookmark' and 'url' get %common.attrs;. When they are really being used this will automatically raise the question: Where to put the value for a common attribute? On a 'bookmark' or on the contained 'url'? Previously we removed the id attribute from bookmark to avoid this. It seems that the 'bare' url is causing subtle problems. Maybe this was a bad decision. Should we undo that and merge url with bookmark? It doesn't cause a big upset since most of bookmarks content is optional anyway. So maybe like this: A minimal bookmark can still be relatively simple: .. Metadata I can follow the reasons for removing ID from metadata. But the ability to reference a block of metadata is now lost. I wonder of this is bad? It is a little more complex but more powerful to be able to reference an info-nugget (it could be done by copying via an entity reference though). Not all info follows the folder hierarchy. On the other hand many cross-refs makes processing more difficult. I'm not sure. Scheme What is the content of the scheme attribute? Should it be an URL (or URN) or can it be any CDATA string? Since an xbel probably uses only a few schemes (relative to the number of bookmarks) but may need to use them in many nodes. It could be worthwhile to declare these in the external subset. (as a notation, or entity-name) or use yet another ID/IDREF pair (or CDATA link attributes) and adding a ) somewhere near the top of the document. This clearly documents which info schemes are being used. Others may exist inside the document but the ones mentioned can be used by reference (this could cut down on the file size in the case of formal scheme names, which tend to be quite long). Other linking issues (maybe consider these for next version) Are there other linking issues? What about a way to make xref's? Link to external xbel documents and/or external metadata information-nuggets? Cheers, Marc -- Marc van Grootel bwaumg@urc.tue.nl From Fred L. Drake, Jr." References: <199810021505.RAA17786@asterix.urc.tue.nl> Message-ID: <13845.1172.752823.623563@weyr.cnri.reston.va.us> --UK/cpViHvX Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Marc van Grootel writes: > Just before the weekend I've got a few possible issues for the XBEL > DTD. I say possible because it's not my intention to upset the current > design. After reading some stuff about metadata I can see that it's a Ha, you're too late! He, he, he.... oh, well. Andrew, please ignore the public text I sent for XBEL this morning. ;-) I'll attach an updated DTD below for continued discussion. > A while ago I suggested adding the ISO Latin 1 entities (like HTML > does) was that ruled out? It would keep XBEL more readable. Do we want just latin-1, or all of the standard ISO entities that have been defined for XML? This would be closer to what HTML uses. I propose to include all those listed at if we're going to allow more than a minimal set. (Note that five currently in previous version of the DTD are defined in the "Numeric and Special Graphic" entity set, not latin-1.) > Folders > > The name for the folder element is directly derived from common > bookmark files. In some ways the 'folder' is like the 'sect' in the > DocBook DTD. One interpretation of a folder is 'a grouped set of > nodes' the fact that this is rendered as a real folder in a bookmark > file is a presentational aspect. I see a real advantage when I can use > a folder element as just a group of nodes inside the current folder > without always being rendered as a separate folder. Such a ... I'm not sure I see the value in what's been defined largely as an interchange format. Most applications would not understand or care about the distinction (assuming I understand it). Perhaps "groups" can be defined using application-specific metadata? HyperTeX FAQ hypertext resources > shortcuts). However there's an asymmetry: an 'url' alias does not > point to an element with info, a 'folder' alias does. So resolving the > 'url' alias (including the info) is now different from resolving a > 'folder' alias. This is a result from our decision to accept 'bare' > url's. This moved the id attribute from bookmark to the url element. This will probably be a non-issue for many bookmark-specific applications, but might create problems for general XML tools. In my current implementation, the internal node created for a is the same as that created for a , and all the right stuff happens by magic. Since the location of attribtutes is currently (and appropriately) constrained, this doesn't present any real issues. Also, note that an can refer to a that doesn't have an child at all, so the argument doesn't appear compelling. > Now both 'bookmark' and 'url' get %common.attrs;. When they are really > being used this will automatically raise the question: Where to put > the value for a common attribute? On a 'bookmark' or on the contained > 'url'? Previously we removed the id attribute from bookmark to avoid > this. > > It seems that the 'bare' url is causing subtle problems. Maybe this > was a bad decision. Should we undo that and merge url with bookmark? > It doesn't cause a big upset since most of bookmarks content is My current implementation attempts to "minimize" the generate output by using a bare if it doesn't cause a loss of information. What I find by looking at the output for my general bookmarks is twofold: (1) most entries turn into elements, which are substantially more compact for viewing by a human, and (2) I've lost a lot of descriptions by testing older versions of Grail bookmark code on "live" data. ;-( I think (hope?) my point is that brevity of markup is pretty valuable for this application. Perhaps should be retained for this reason, and perhaps not. There is good thinking in putting "id" on the element, however, and using just one element would simplify processing somewhat. But let's take a look at the size difference anyway, since I've brought it up. This uses the previous version of XBEL: HyperTeX FAQ This is the same bookmark, without : HyperTeX FAQ Well, that doesn't look so bad. I'll go ahead and adjust the DTD. > Metadata > > I can follow the reasons for removing ID from metadata. But the > ability to reference a block of metadata is now lost. I wonder of this I see two real options: put "id" attributes only on things that it make immediate sense to refer to via , and to put it in common.attrs. The only place to link to anything other than folders and bookmarks (assuming that linking to a folder makes sense), is from outside the document. I would expect this to happen rarely, if ever. From this perspective, I'll vote for simplicity. > It is a little more complex but more powerful to be able to > reference an info-nugget (it could be done by copying via an entity > reference though). Not all info follows the folder hierarchy. On the Perhaps the need to linking will be made clear if you can describe an application of it? > Scheme > > What is the content of the scheme attribute? Should it be an URL (or > URN) or can it be any CDATA string? Since an xbel probably uses only CDATA seems appropriate; there's no catalog of metadata schemes. Since is overloaded with application-specific schemes, we cannot presume to predict the range of possible values. I just took a look at the element from HTML 4.0, and it looks like there's a slightly different approach described there: has a "profile" attribute which names what I've been calling the scheme, and the attribute "scheme" is used to describe the notation of the metadata value. Perhaps the "scheme" should be named "profile", and add an HTML 4.0-style "scheme" to , primarily to allow applications which collect information from HTML pages to store it without loosing fidelity. > ID/IDREF pair (or CDATA link attributes) and adding a name="a-long-formal-id" id="s1"/>) somewhere near the top of the > document. This clearly documents which info schemes are being > used. Others may exist inside the document but the ones mentioned can > be used by reference (this could cut down on the file size in the case > of formal scheme names, which tend to be quite long). I'd avoid this since I don't know of any formal registries for metadata schemes/profiles/whatever. This can also be accomplished using general entities: ]> HyperTeX FAQ ... > Other linking issues (maybe consider these for next version) > > Are there other linking issues? What about a way to make xref's? Link > to external xbel documents and/or external metadata > information-nuggets? There comes a point at which we punt and require people to use XPointer. Or wait for specific issues to crop up and make a new version. ;-) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 --UK/cpViHvX Content-Type: text/xml Content-Description: YAX (Yet Another XBEL) Content-Disposition: inline; filename="xbel.dtd" Content-Transfer-Encoding: 7bit --UK/cpViHvX-- From Fred L. Drake, Jr." I'm not attaching another copy of the DTD, but I'd like to make the element optional within <bookmark>; for many resources (such as FTP directories), there really isn't a real title, so software should have the option of omitting it. It should still be o.k. to generate a title, but an omitted title should also be acceptable. The <bookmark> "href" can be used as a stand-in if needed. For some URIs, it may make sense for software that displays bookmarks to generate a title on-the-fly anyway. In particular, for a URN scheme like ietf:, a generic title could be generated in the local language. For example: href="ietf:rfc:822" could get a generated title of "Internet Request for Comments 822" for English users. Other URN schemes may also provide ways of looking up language-specific titles. Software which provides title-retrieval on an as-appropriate basis could also be configured to use retrieval instead of using the provided title, so if anyone has a problem with making the title optional, I can probably live with it. -Fred -- Fred L. Drake, Jr. <fdrake@acm.org> Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From gstein@lyra.org Fri Oct 2 23:54:43 1998 From: gstein@lyra.org (Greg Stein) Date: Fri, 02 Oct 1998 15:54:43 -0700 Subject: [XML-SIG] Proposed XBEL DTD References: <199810021505.RAA17786@asterix.urc.tue.nl> <13845.1172.752823.623563@weyr.cnri.reston.va.us> Message-ID: <361559B3.5F73BF80@lyra.org> Okay guys... here comes another swipe at complexity :-) Fred L. Drake wrote: > ... > > A while ago I suggested adding the ISO Latin 1 entities (like HTML > > does) was that ruled out? It would keep XBEL more readable. > > Do we want just latin-1, or all of the standard ISO entities that > have been defined for XML? This would be closer to what HTML uses. > I propose to include all those listed at > <http://www.schema.net/entities/> if we're going to allow more than a > minimal set. (Note that five currently in previous version of the DTD > are defined in the "Numeric and Special Graphic" entity set, not > latin-1.) I think you guys are making this overly complicated. I don't see any DTD out there that specifies additional entities. Instead, standard character encoding is used. I seem to recall reading somewhere that part of the purpose of XML was to get rid of all the random entities that HTML had (which were generally not universally recognized anyhow), and to limit the entities down to just a handful. Those handful of entities are builtin to XML parsers and do not require specification in a DTD. (lt, gt, etc) Punting this whole entity thing seems to make a lot of sense to me. If machines are the primary users of XBEL, then you won't be using entities anyhow, in lieu of standard encoding. > > Folders > > > > The name for the folder element is directly derived from common > > bookmark files. In some ways the 'folder' is like the 'sect' in the > > DocBook DTD. One interpretation of a folder is 'a grouped set of > > nodes' the fact that this is rendered as a real folder in a bookmark > > file is a presentational aspect. I see a real advantage when I can use > > a folder element as just a group of nodes inside the current folder > > without always being rendered as a separate folder. Such a > .. > > I'm not sure I see the value in what's been defined largely as an > interchange format. Most applications would not understand or care > about the distinction (assuming I understand it). Perhaps "groups" > can be defined using application-specific metadata? > > <bookmark href="http://xxx.lanl.gov/hypertex/" > added="1998-01-27T13:09:45-05:00" > visited="1998-03-24T10:12:10-05:00"> > <title>HyperTeX FAQ > > > hypertext resources > > > Can we name or define an application before adding it? Why put in complexity if you don't have an outline need? > ... > > Now both 'bookmark' and 'url' get %common.attrs;. When they are really > > being used this will automatically raise the question: Where to put > > the value for a common attribute? On a 'bookmark' or on the contained > > 'url'? Previously we removed the id attribute from bookmark to avoid > > this. I believe the whole %common.attrs thing is bogus. It appears its only purpose in the DTD is to make it more complicated. It doesn't define any common attributes, and I bet nobody can come up with one that is common across ALL elements in there (it HAS been applied to all elements, after all). What's the purpose? Torch it. Don't say "for the future" ... add it in the future if you find you need it. > > > > It seems that the 'bare' url is causing subtle problems. Maybe this > > was a bad decision. Should we undo that and merge url with bookmark? > > It doesn't cause a big upset since most of bookmarks content is > ... > Well, that doesn't look so bad. I'll go ahead and adjust the DTD. Woo hoo! Simplification :-) > > Metadata > > > > I can follow the reasons for removing ID from metadata. But the > > ability to reference a block of metadata is now lost. I wonder of this > > I see two real options: put "id" attributes only on things that it > make immediate sense to refer to via , and to put it in > common.attrs. The only place to link to anything other than folders > and bookmarks (assuming that linking to a folder makes sense), is from > outside the document. I would expect this to happen rarely, if ever. > >From this perspective, I'll vote for simplicity. I believe I missed the whole rationale for to begin with. What is its intended purpose? Maybe some comments could be inserted into the DTD to describe the general usage/purpose of the different elements? >... > > Scheme > > > > What is the content of the scheme attribute? Should it be an URL (or > > URN) or can it be any CDATA string? Since an xbel probably uses only > > CDATA seems appropriate; there's no catalog of metadata schemes. > Since is overloaded with application-specific schemes, we > cannot presume to predict the range of possible values. I just took a Ah, but I bet you can say that whatever it is, it must be unique so that an application can differentiate its schemes/profiles from another. I'd argue for the simplicity and uniqueness of a URL. If you make it a CDATA, then people need to ask "what is the typical format? how do I ensure uniqueness?" And they'll just stick a URL in there anyhow. > ... > > ID/IDREF pair (or CDATA link attributes) and adding a > name="a-long-formal-id" id="s1"/>) somewhere near the top of the > > document. This clearly documents which info schemes are being > > used. Others may exist inside the document but the ones mentioned can > > be used by reference (this could cut down on the file size in the case > > of formal scheme names, which tend to be quite long). > > I'd avoid this since I don't know of any formal registries for > metadata schemes/profiles/whatever. This can also be accomplished > using general entities: > > > ]> > > added="1998-01-27T13:09:45-05:00" > visited="1998-03-24T10:12:10-05:00"> > HyperTeX FAQ > > > ... > > > Actually, this begs the whole question: what the heck are metadata elements doing in there anyhow? Why not use arbitrary XML elements (with their potential for a namespace)? Your example above looks suspiciously like namespaces. Here is an example: ... I'd say punt the whole metadata thing and rely on applications to define their own XML elements and place those into the area. > > > Other linking issues (maybe consider these for next version) > > > > Are there other linking issues? What about a way to make xref's? Link > > to external xbel documents and/or external metadata > > information-nuggets? > > There comes a point at which we punt and require people to use > XPointer. Or wait for specific issues to crop up and make a new > version. ;-) yah yah! :-) Complexity is generally the reciprocal of usefulness. Some of the complexity that is bulking it up (e.g. common.attrs) almost appears to be like peacock feathers. :-) Keep it simple... people will use it. Cheers, -g -- Greg Stein (gstein@lyra.org) From digitome@iol.ie Sat Oct 3 09:42:46 1998 From: digitome@iol.ie (Sean Mc Grath) Date: Sat, 03 Oct 1998 09:42:46 +0100 Subject: [XML-SIG] Proposed XBEL DTD In-Reply-To: <361559B3.5F73BF80@lyra.org> References: <199810021505.RAA17786@asterix.urc.tue.nl> <13845.1172.752823.623563@weyr.cnri.reston.va.us> Message-ID: <3.0.6.32.19981003094246.00954100@gpo.iol.ie> [Greg Stein] >I think you guys are making this overly complicated. I don't see any DTD >out there that specifies additional entities. Instead, standard >character encoding is used. I agree with Greg on this point. > >I seem to recall reading somewhere that part of the purpose of XML was >to get rid of all the random entities that HTML had (which were >generally not universally recognized anyhow), and to limit the entities >down to just a handful. Those handful of entities are builtin to XML >parsers and do not require specification in a DTD. (lt, gt, etc) Yes. For compatibility with SGML, the spec. suggests that DTDs define the entities that XML builds in but is SGML compatability really important for XBEL? And even if it is, will SGML users be put out? I think not. I use SGML a lot and not having the entities declared in XBEL does not worry me in the least. The pre-eminent SGML parser SP will infer them automatically if you ask it to parse XML as SGML. I suspect the same will be true of all SGML tools in the future. As for the iso entity sets, most of the need for these things goes out the window with XML as Unicode is the character set. Yes, you still need entities for things like chemical symbols and so on but must human language symbols are dealt with by Unicode (including Klingon:-). Looking to the future a bit:- I can see the entire entity architecture of XML being depracated in favour of transclusion etc. via XLink. I can see DTDs becoming a "compatability only" way of doing schemas. 9 out of 10 XBEL aware apps will use non-validating XML parsing! Sean Mc Grath def Get_URI_Of_Superlative_Scripting_Language(): return "http://www.python.org" From fredrik@pythonware.com Sat Oct 3 16:28:57 1998 From: fredrik@pythonware.com (Fredrik Lundh) Date: Sat, 3 Oct 1998 17:28:57 +0200 Subject: [XML-SIG] Proposed XBEL DTD Message-ID: <02b701bdeee2$8a6b2860$f29b12c2@pythonware.com> > PUBLIC "ISO 8879:1986//ENTITIES Added Latin 1//EN//XML" > "http://www.schema.net/public-text/ISOlat1.pen"> Watching how XBEL evolved from my first little hack to a complex beast with external entity definitions, metadata stuff, and lots of %foo.attrs stuff makes me realize why there's so much XML confusion out there... (Even if you think you understand exactly what a browser bookmark is, that doesn't mean that you can quickly figure out how to use a simple DTD like XBEL, not to mention understanding anything written by an XML expert ;-) Afaik, the XML standard already describes how to deal with various character sets. And since this format is basically designed for computer consumption, adding tons of entities doesn't feel necessary. Please keep this as simple as it possibly can be! Simple-minded-ly yrs /F From bwaumg@urc.tue.nl Sun Oct 4 14:56:16 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Sun, 04 Oct 1998 15:56:16 +0200 Subject: [XML-SIG] XBEL: bookmark title Message-ID: <199810041356.PAA24509@asterix.urc.tue.nl> Fred L. Drake, Jr. writes: > ... > so if anyone has a problem with making the title > optional, I can probably live with it. fine with me Marc -- Marc van Grootel bwaumg@urc.tue.nl From bwaumg@urc.tue.nl Sun Oct 4 14:57:45 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Sun, 04 Oct 1998 15:57:45 +0200 Subject: [XML-SIG] Proposed XBEL DTD Message-ID: <199810041357.PAA24565@asterix.urc.tue.nl> Greg Stein wrote: > Complexity is generally the reciprocal of usefulness. Some of the > complexity that is bulking it up (e.g. common.attrs) almost appears to > be like peacock feathers. :-) Keep it simple... people will use it. Greg Stein wrote: > I believe the whole %common.attrs thing is bogus. It appears its only > purpose in the DTD is to make it more complicated. It doesn't define any > common attributes, and I bet nobody can come up with one that is common > across ALL elements in there (it HAS been applied to all elements, after > all). I agree on the common.attrs issue, and partly on the complexity issue. common.attrs looks like a hook for future extension of the DTD. However, I think that currently it is not needed, and because it is attached to many elements could cause problems when it will actually be used. Secondly, generally, parameter entities are used to make the DTD clearer (more self documenting) not more complex. What kind of complexity are we talking about here? The perceived complexity of a DTD says little about the complexity of the average document instance. Maybe all the special entity sets can be left out. But what harm is there in explicitly declaring those few entities (<,>,&,',") for interoperability? Even the spec suggests it. Sean Mc Grath wrote: > Looking to the future a bit:- > > > I can see the entire entity architecture of XML being depracated > in favour of transclusion etc. via XLink. > > I can see DTDs becoming a "compatability only" way of doing schemas. > > 9 out of 10 XBEL aware apps will use non-validating XML parsing! > > Greg Stein wrote: > Actually, this begs the whole question: what the heck are metadata > elements doing in there anyhow? Why not use arbitrary XML elements (with > their potential for a namespace)? Your example above looks suspiciously > like namespaces. Here is an example: > > > > .. > > > > > > > I'd say punt the whole metadata thing and rely on applications to define > their own XML elements and place those into the area. To me these are important points. I hate to let SGML compatibility and the possibility of validating against a DTD go. Especially given the status of XML namespaces versus SGML. Maybe I'm just having troubles adapting to XML. It may not be heresy but after a while people may indeed say "why the hell bother with DTD's they're a nuisance". At that point others will start getting a severe case of deja-vu as they remember what happened to HTML and nostalgically haul out a dusty manual with that XML 1.0 specification where it says: "XML shall be compatible with SGML" and "The number of optional features in XML is to be kept to the absolute minimum, ideally zero". But that's just my $0.02 Marc -- Marc van Grootel bwaumg@urc.tue.nl From larsga@ifi.uio.no Sun Oct 4 16:37:14 1998 From: larsga@ifi.uio.no (Lars Marius Garshol) Date: 04 Oct 1998 16:37:14 +0100 Subject: [XML-SIG] Proposed XBEL DTD In-Reply-To: <3.0.6.32.19981003094246.00954100@gpo.iol.ie> References: <199810021505.RAA17786@asterix.urc.tue.nl> <13845.1172.752823.623563@weyr.cnri.reston.va.us> <3.0.6.32.19981003094246.00954100@gpo.iol.ie> Message-ID: * Sean Mc Grath | | I can see DTDs becoming a "compatability only" way of doing schemas. | | 9 out of 10 XBEL aware apps will use non-validating XML parsing! Both these points are valid, but IMHO the XBEL DTD is still a good idea because it concisely and formally defines XBEL and at the same time provides a means of validating XBEL documents. Other than that I definitely agree with the 'keep it simple'-camp. --Lars M. From larsga@ifi.uio.no Sun Oct 4 20:15:03 1998 From: larsga@ifi.uio.no (Lars Marius Garshol) Date: 04 Oct 1998 20:15:03 +0100 Subject: [XML-SIG] Some points & ideas Message-ID: I'm going away for the next month and will be offline for at least two weeks to come, so I'm posting some thoughts here before I log off. --- saxlib Two problems have shown up: - Jack Jansen unearthed a bug in saxutils.ErrorPrinter - Paul Prescod pointed out that EntityResolver.resolveEntity should return a file-like object instead of the system identifier. (In the Java version it returns an InputSource.) I've also been thinking that once we have support for different character sets (whether in Python itself or the parsers) we will need InputSource. This clashes with my wish to keep saxlib 1.0 untouched until SAX 1.0 itself changes, which still seems to be a long way off. Any thoughts/opinions on this? Does anyone use the resolveEntity method enough to be upset by a modification to it? Anyone against InputSource being added to saxlib? --- More functionality Something like John Cowan's ParserFilters for SAX would be nice to have for saxlib and also very simple to make. See What do people think, do we want this? Anyone feel like making it? --- Even more functionality A validation package that used regular expressions to express constraints on element content and attribute values, as well as predicate functions (identified by module.function) to validate documents. This would be nice to have and should be easy to implement as a ParserFilter. Any comments on this? That's just some of what I had in my head right now. I'll be going offline in about 2-3 hours. --Lars M. From akuchlin@cnri.reston.va.us Sun Oct 4 23:25:19 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Sun, 4 Oct 1998 18:25:19 -0400 Subject: [XML-SIG] DOM code now in CVS tree Message-ID: <199810042225.SAA01726@207-172-112-102.s102.tnt4.ann.erols.com> I have (finally) checked in the DOM code into the anonymous CVS repository. Some random notes: ================== * I am definitely not claiming completeness. There are objects and sections of code that have never been executed yet. But it is possible to build a little DOM tree. Wide string support isn't in there, and won't be until Python supports it. * Attributes in the IDL DOM definition are implemented as methods, prefixed with get_. For example, to retrieve the childNodes attribute, you call .get_childNodes() . This is not an optimal solution. Ideally it would be as simple as accessing .childNodes, but this faces some problems. First, the low-level tree of elements is implemented as a tree of _nodeData instances, with no circular references. Element, Text, etc. instances are then implemented as proxies for the corresponding _nodeData instance. This provides two useful features: 1) If there are several proxies for a given _nodeData, changes are instantly visible to all of the proxies. 2) As Ken McLeod suggested, this should avoid circular references, and, therefore, memory leaks. (I haven't actually verified that there are no leaks, though.) However, this also means that just setting up static childNodes attributes would break property 1. So you'd have to write __getattr__/__setattr__ functions. I'm worried that this would exact too great a performance penalty. Thoughts? * You can add extended interfaces and still remain compliant with the Recommendation. Therefore, feel free to suggest other useful things. I've added a few convenient aliases: the DOM spec specifies get_nodeName() and get_nodeValue(), so I've added get_name() and get_value(). We can add other alises to fix other annoyances. * Not all the stuff in the dom/ subdirectory has been fixed to match the current DOM interfaces. walker.py, writer.py, builder.py, and sax_builder.py have been quickly patched up, but they haven't been tested much. * Following the responses to my query about preserving compatibility, there's no attempt to preserve compatibility with the earlier DOM code at all. * There's no attempt yet at representing DTD information. Some degree of this must be done, however, in order to handle default attribute values. (If an attribute of an element has a default value, then you can override its value by assigning to it; if you then delete that attribute, the default value springs back.) * How should we linearise a DOM tree? Currently the __str__ method sort of does this; however, if the text is '>', should str(TextNode) return '>' or '>'? It's probably a better idea to have __repr__ convert nodes to a helpful description of the object (for debugging), and __str__ will return node.get_nodeData() for some node types such as Element and Comment; other nodes would simply have __str__ the same as __repr__. A third method could then linearise a node and its descendants; this would also allow adding some method arguments to control whether the XML produced is pretty-printed, indentation parameters, etc. Or, we can keep this in another class, as it is now; I'd prefer to have it be available in the core, though. Where do we go now? =================== * I hope that people will grab the latest CVS snapshot, and try the code out, reporting bugs and non-compliant points. Bonus points for providing a patch. * In the meantime, I'm going to work on adding docstrings to the code, implement the few missing things, fix any reported bugs, and begin work on a DOM test suite (which is going to be a sizable job). * Once the DOM code has been shaken down for a while, it'll be time for a 0.5 release. If no one tries the DOM code, the 0.5 release may be delayed for quite a while, so please try it out. Oh, yes; some sample code... from xml.dom import core doc = core.createDocument() html = doc.createElement('html') html.setAttribute('attr', 'value') head = doc.createElement('head') title = doc.createElement('title') title.appendChild( doc.createTextNode("Title goes here") ) body = doc.createElement('body') body.setAttribute('background', '#ffffff') comment = doc.createComment("Comment between head and body") doc.appendChild(html) html.appendChild(head) head.appendChild(title) html.appendChild(body) print doc This outputs: Title goes here (No DTD info yet, remember.) -- A.M. Kuchling http://starship.skyport.net/crew/amk/ When a man tells you that he got rich through hard work, ask him *whose*? -- Don Marquis From gstein@lyra.org Mon Oct 5 00:05:53 1998 From: gstein@lyra.org (Greg Stein) Date: Sun, 4 Oct 1998 16:05:53 -0700 Subject: [XML-SIG] DOM code now in CVS tree In-Reply-To: <199810042225.SAA01726@207-172-112-102.s102.tnt4.ann.erols.com> Message-ID: <001601bdefeb$88fb8560$0e2bc0d0@suntzu.lyra.org> Andrew wrote: > * Attributes in the IDL DOM definition are implemented as > methods, prefixed with get_. For example, to retrieve the childNodes > attribute, you call .get_childNodes() . > > This is not an optimal solution. Ideally it would be as > simple as accessing .childNodes, but this faces some problems. First, > the low-level tree of elements is implemented as a tree of _nodeData > instances, with no circular references. Element, Text, etc. instances > are then implemented as proxies for the corresponding _nodeData > instance. This provides two useful features: > > 1) If there are several proxies for a given _nodeData, > changes are instantly visible to all of the proxies. > 2) As Ken McLeod suggested, this should avoid circular > references, and, therefore, memory leaks. (I haven't > actually verified that there are no leaks, though.) > > However, this also means that just setting up static > childNodes attributes would break property 1. So you'd have to write > __getattr__/__setattr__ functions. I'm worried that this would exact > too great a performance penalty. Thoughts? In both cases, you're executing a method, so I don't see that there would be much of a perf difference. Go for __getattr__. > * You can add extended interfaces and still remain compliant > with the Recommendation. Therefore, feel free to suggest other useful > things. I've added a few convenient aliases: the DOM spec specifies > get_nodeName() and get_nodeValue(), so I've added get_name() and > get_value(). We can add other alises to fix other annoyances. You could always create subclasses of the DOM code for the Python-extended versions. Have a "pure" version and one with the additional Python interfaces. This goes along with the getattr from above. Cheers, -g -- Greg Stein (gstein@lyra.org) From ekhera@cs.monash.edu.au Mon Oct 5 05:41:03 1998 From: ekhera@cs.monash.edu.au (Eklavya Khera) Date: Mon, 5 Oct 1998 14:41:03 +1000 (EST) Subject: [XML-SIG] Ignoring '<' and '>' in SGML In-Reply-To: <199809261600.MAA02602@python.org> Message-ID: Hi, I have found a way of ignoring the '<' and '>' in HTML, and it just printd them out, rather than interpreting them as tags. I was wondering if this could be done in some way in SGML as I want to put in some sample code, that it should just display without giving me errors and telling me they are not tags ... Thanks, Eklav From bwaumg@urc.tue.nl Mon Oct 5 10:26:31 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Mon, 05 Oct 1998 11:26:31 +0200 Subject: [XML-SIG] Ignoring '<' and '>' in SGML Message-ID: <199810050926.LAA19961@asterix.urc.tue.nl> > Hi, > I have found a way of ignoring the '<' and '>' in HTML, and it just printd > them out, rather than interpreting them as tags. > I was wondering if this could be done in some way in SGML as I want to put > in some sample code, that it should just display without giving me errors > and telling me they are not tags ... > Thanks, > Eklav > in SGML you can declare an element having CDATA content: However this is not a solution for putting markup inside it since the parser will still look for ' this will be left alone by the parser ]]> This should work for XML and SGML. Of course you could also encode the markup with <foo> etc. Marc -- Marc van Grootel bwaumg@urc.tue.nl From Fred L. Drake, Jr." References: <199810021505.RAA17786@asterix.urc.tue.nl> <13845.1172.752823.623563@weyr.cnri.reston.va.us> <361559B3.5F73BF80@lyra.org> Message-ID: <13848.61944.53403.175010@weyr.cnri.reston.va.us> Greg Stein writes: > I think you guys are making this overly complicated. I don't see any DTD > out there that specifies additional entities. Instead, standard > character encoding is used. Ok, they're out. I'll even leave out lt, gt, amp, apos, and quot this time around. (Note that for something that's more "document oriented" and edited directly by humans, I'd argue a lot more, but I don't expect there will be much human editing of bookmark files. ;) > I seem to recall reading somewhere that part of the purpose of XML was > to get rid of all the random entities that HTML had (which were > generally not universally recognized anyhow), and to limit the entities I don't remember that, and don't feel that broken HTML processing software is terribly relevant. > Punting this whole entity thing seems to make a lot of sense to me. If > machines are the primary users of XBEL, then you won't be using entities > anyhow, in lieu of standard encoding. *This* makes a lot of sense, though. > Can we name or define an application before adding it? Why put in > complexity if you don't have an outline need? Isn't that what Marc did? His application is to support non-hierarchical groups as well as hierarchical groups. *Why* he'd want to do that is his problem. ;-) > I believe the whole %common.attrs thing is bogus. It appears its only > purpose in the DTD is to make it more complicated. It doesn't define any > common attributes, and I bet nobody can come up with one that is common > across ALL elements in there (it HAS been applied to all elements, after More experienced DTD designers than myself developed the idiom. The basic thinking behind it reflects the same thought on the topic that I have: this allows "subclassing" the DTD without modifying the DTD itself. This can be done by defining the appropriate parameter entities with the new values, and then including the original DTD. This makes it easier to reuse the existing DTD and integrate extensions appropriately. I'm more inclined to work on this a little more, and improve the comments / documentation accordingly. One important aspect to remember is that the use of parameter entites does *not* make the DTD more complex; putting something *in* them does. > I believe I missed the whole rationale for to begin with. What > is its intended purpose? The purpose of is to support Netscape's bookmark aliasing capability. I played with that the other day; it turns out that only bookmarks can be aliased, not folders, at least in 4.04. This means we can remove "id" from and have it only on . (Done in the attached DTD.) I'll clarify this in a comment for now; it will be discussed in the documentation. > Ah, but I bet you can say that whatever it is, it must be unique so that > an application can differentiate its schemes/profiles from another. I'd > argue for the simplicity and uniqueness of a URL. If you make it a > CDATA, then people need to ask "what is the typical format? how do I > ensure uniqueness?" And they'll just stick a URL in there anyhow. Yes, URIs (including URLs) are CDATA. I'll add a comment about it. (Requiring only that it be a URI and not necessaily a URL is sufficient for uniqueness, and supports more communities; not everyone thinks URLs are all that useful for long-term addressing.) > I'd say punt the whole metadata thing and rely on applications to define > their own XML elements and place those into the area. Marc, you argued for having an explicit metadata structure; do you want to respond to this? -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From Fred L. Drake, Jr." References: <199810021505.RAA17786@asterix.urc.tue.nl> <13845.1172.752823.623563@weyr.cnri.reston.va.us> <361559B3.5F73BF80@lyra.org> <3.0.6.32.19981003094246.00954100@gpo.iol.ie> Message-ID: <13848.62862.306979.380983@weyr.cnri.reston.va.us> Sean Mc Grath writes: > [Greg Stein] > >I think you guys are making this overly complicated. I don't see any DTD > >out there that specifies additional entities. Instead, standard > >character encoding is used. > > I agree with Greg on this point. Ok, the DTD has been adjusted. > the entities that XML builds in but is SGML compatability really > important for XBEL? And even if it is, will SGML users be put out? SGML compatibility is not important for XBEL. Now that I've gone back and checked to see just what "for compatibility" means, I'm leaving out the definitions of gt, lt, etc. as well. > > I can see the entire entity architecture of XML being depracated > in favour of transclusion etc. via XLink. Possibly. It's not here yet. > I can see DTDs becoming a "compatability only" way of doing schemas. Possibly. But we don't have any alternatives today. > 9 out of 10 XBEL aware apps will use non-validating XML parsing! > Yes. But does this matter? It needs to be possible to validate XBEL, if only so implementors can ensure their output is usable by others. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From Fred L. Drake, Jr." --cwfcFncvGF Content-Type: text/plain; charset=us-ascii Content-Description: message body text Content-Transfer-Encoding: 7bit Ok, I forgot to attach the XBEL DTD after my response to Greg's comments, so here it is. What's changed? Only carries the "id" attribute, so only elements may be referred to by elements. Comments were added to help explain "scheme" and . Reference to the ISO general entities has been removed. There are still outstanding issues related to the metadata stuff. I'm waiting for commentary from Marc before I make changes there. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 --cwfcFncvGF Content-Type: text/xml Content-Description: YAX (Yet Another XBEL) Content-Disposition: inline; filename="xbel.dtd" Content-Transfer-Encoding: 7bit --cwfcFncvGF-- From stevenn@xs4all.be Mon Oct 5 21:50:10 1998 From: stevenn@xs4all.be (Steven Noels) Date: Mon, 5 Oct 1998 22:50:10 +0200 Subject: [XML-SIG] Annual SGML BeLux conference, version 1998 Message-ID: <004701bdf0a1$beb9f220$82a1fea9@titus.xs4all.be> Sorry if this is off-topic: ------------------------------------------------------------------------ SGML BeLux Conference 1998, October 21st, Antwerp ------------------------------------------------------------------------ SGML BeLux' 5th annual conference on the practical use of SGML/XML XML & SGML in action. The only two-day SGML/XML event of its kind that shows you how to use standard markup languages to bring your document and data processing at a higher level. New! Introduction Seminars on SGML and XML Special focus: Applications in the field of linguistic engineering and multilinguality Conference Rooms VEV/SD WORX/ZENO, Brouwersvliet, Antwerp 20th and 21st of October, 1998 ------------------------------------------------------------------------ You can find more information, even an online registration form at http://www.sgmlbelux.be/conf98/ We've exceeded our own expectations this year, so please take a look at our conference program (also on the website). During conference breaks, lunch and reception (to finish the day), you can meet with most of the leading SGML/XML tools vendors, which are participating with our exhibition. Hope to see you there! From bwaumg@urc.tue.nl Tue Oct 6 00:45:09 1998 From: bwaumg@urc.tue.nl (Marc van Grootel) Date: Tue, 06 Oct 1998 01:45:09 +0200 Subject: [XML-SIG] Proposed XBEL DTD Message-ID: <199810052345.BAA08444@asterix.urc.tue.nl> > Marc, you argued for having an explicit metadata structure; do you > want to respond to this? Not quite, Fred. I proposed the meta tag approach. You came up with metadata as a wrapper for multiple meta elements. I liked it. I believe that info like it is can extend the lifetime of the DTD, and it can also be an escape-hatch for new bookmark apps. I guess the need for some sort of metadata is apparent, it was there from the start when Mark Hammond suggested: How does this merger of namespaces with the info element look? dma bar 10 We can claim the xbel namespace and put essential info stuff in it. I regret losing id from folder. Why is an alias to a folder stranger then an alias to a bookmark. NS has the latter but not the first. MSIE supports both. NS has a separator, MSIE doesn't know such a thing, still, separator is in. Why does NS get to veto elements/attributes? G'night Marc --- Marc van Grootel bwaumg@urc.tue.nl From akuchlin@cnri.reston.va.us Tue Oct 6 03:08:49 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Mon, 5 Oct 1998 22:08:49 -0400 Subject: [XML-SIG] DOM code update Message-ID: <199810060208.WAA02796@207-172-42-133.s133.tnt17.ann.erols.com> Fred Drake tried out the DOM code and pointed out various problems. Monday evening's activity, now checked into CVS: Reindented the body of the Node class. Renamed the .index method to _index Added some more __repr__ methods. Fix the insertBefore(), appendChild(), and replaceChild() method to delete the node being added when it's already present in the tree. Rename the __str__() functions to toxml(); this will be how you convert a DOM tree or subtree to XML. (The method name may still change.) Fixed getElementsByTagName(); it actually works now. Fixed typo in ProcessingInstruction.toxml(); PIs end with ?> Implemented normalize() and splitText(). Delete a Linux binary in the expat/ tree which messed up building on Solaris. Deleted the top-level Makefile from CVS; Makefile.pre.in should build it. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ As always, I'll leave it to a volunteer to experiment with this. -- Guido van Rossum, 20 Jan 1995 From Fred L. Drake, Jr." References: <199810052345.BAA08444@asterix.urc.tue.nl> Message-ID: <13850.13313.413513.303143@weyr.cnri.reston.va.us> Marc van Grootel writes: > > Marc, you argued for having an explicit metadata structure; do you > > want to respond to this? > > Not quite, Fred. I proposed the meta tag approach. You came up with > metadata as a wrapper for multiple meta elements. I liked it. I Marc, Sorry; I wasn't trying to saddle you with the specific structure, just that there was an explicit structure. > believe that info like it is can extend the lifetime of the DTD, and > it can also be an escape-hatch for new bookmark apps. I guess the need > for some sort of metadata is apparent, it was there from the start ... > > dma > bar ... > We can claim the xbel namespace and put essential info stuff in it. We cannot "claim" namespaces; namespaces are declared using processing instructions, as indicated in Greg's example. Namespaces don't need to be changed to be usable; they actually fit the need fairly well, but I'm not convinced they're exactly what's needed by themselves. I propose that remains as a sequence of elements, each of which uses an attribute to indicate an application or scheme of some sort. can be declared to hold #PCDATA or whatever, and then namespaces are used to describe the structure of the contents. This would allow multiple applications to use the same DTD as the source of their information (hence the same namespace), while the distinction between applications is made clear. (It's entirely possible that multiple RDF schemes would be stored, so ownership of various chunks may well need to be identified separately from the namespace specifications. > I regret losing id from folder. Why is an alias to a folder stranger > then an alias to a bookmark. NS has the latter but not the first. MSIE > supports both. NS has a separator, MSIE doesn't know such a thing, I did not know this. I'll add it back. This does introduce the possibility of circular data structures, so applications will need to be careful about traversing aliases to folders. That can be dealt with in the documentation. How does MSIE deal with aliases to folders (including in the user interface)? I think the dominant requirement is that XBEL be sufficient for storing all information currently stored in browser bookmarks for at least the major browsers, and preferably minor players as well. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Tue Oct 6 17:00:33 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 6 Oct 1998 12:00:33 -0400 (EDT) Subject: [XML-SIG] DOM: Multiple proxy problem Message-ID: <13850.13247.104019.731837@amarok.cnri.reston.va.us> I've realized there's a potential problem with the current DOM implementation, and am wondering how to fix it. Problem: ======== The basic structure of a DOM document tree is stored as a tree of _nodeData instances; each instance contains data about that node (type, value, children, etc.). Instances don't hold a reference to their parent, because this causes cycles which make Python's refcounting GC fail. (I may eventually change _nodeData instances into lists, which may reduce the memory usage and improve the speed. That's irrelevant for the moment.) A user of the DOM interface will never come into contact with a _nodeData instance; instead, the user deals with Element, Text, CDATASection, ... instances, which contain some attributes (including, most importantly, the parent node) and delegates other attributes to the _nodeData instance that's being proxied. This avoids creating cycles, and the garbage collector and user frolic happily. However... consider having multiple proxies for the same _nodeData instance. Here's a tree: Element 'title' Text 'chunk of text' Text 'another chunk' Call the two text nodes t1 and t2. Get two references to t2, by something like: ref1 = Element.get_childNodes()[1] ref2 = Element.get_childNodes()[1] Now, move t2 somewhere else: OtherElement.appendChild( ref1 ) This automatically deletes t2 from its parent's list of children, adds it to the other element, and updates ref1's parent pointer. ref2's pointer is now out of date; if you then do YetAnotherElement.appendChild( ref2 ), things will go funny. Solution (?) ============ How do we solve this? Given a _nodeData instance, there's no fast way to find its parent; you can start at the root and crawl the tree, but this isn't a complete solution, because the node may have been removed from the tree and added to a DocumentFragment. (Can we invent a way of quickly finding the parent, without reintroducing cycles? This would fix the problem nicely, and in fact probably remove the need for proxies completely.) We can, at least, detect the problem; if appendChild finds that t2 isn't actually a child of the purported parent node, it can raise an exception. I'll implement that sanity check in the meantime, and may just leave it in; if people encounter that exception frequently, the problem is worth fixing more carefully. I'd really appreciate hearing some suggestions about this. BTW, using PyExpat on my Ultra 10, the 279K hamlet.xml file is converted to a DOM tree in about 7 seconds; about 40K/sec. The DOM tree seems to be correct, though it's leaving in Text nodes containing ignorable whitespace; got to fix that... -- A.M. Kuchling http://starship.skyport.net/crew/amk/ I didn't say it was my fault. I said it was my responsibility. I know the difference. -- Rose Walker, in SANDMAN #60: "The Kindly Ones:4" From Fred L. Drake, Jr." References: <13850.13247.104019.731837@amarok.cnri.reston.va.us> Message-ID: <13850.19162.682792.888605@weyr.cnri.reston.va.us> Andrew M. Kuchling writes: > tree seems to be correct, though it's leaving in Text nodes containing > ignorable whitespace; got to fix that... What's wrong with this? I should be able to get ignorable writespace nodes as well as regular whitespace nodes. Is there at least an option to get the whole thing, if the parser provides the events? -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Tue Oct 6 18:54:14 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 6 Oct 1998 13:54:14 -0400 (EDT) Subject: [XML-SIG] DOM: Multiple proxy problem In-Reply-To: <13850.19162.682792.888605@weyr.cnri.reston.va.us> References: <13850.13247.104019.731837@amarok.cnri.reston.va.us> <13850.19162.682792.888605@weyr.cnri.reston.va.us> Message-ID: <13850.21875.949008.651441@amarok.cnri.reston.va.us> Fred L. Drake writes: >Andrew M. Kuchling writes: > > tree seems to be correct, though it's leaving in Text nodes containing > > ignorable whitespace; got to fix that... > > What's wrong with this? I should be able to get ignorable >writespace nodes as well as regular whitespace nodes. Is there at >least an option to get the whole thing, if the parser provides the >events? The DOM spec doesn't actually mention ignorable whitespace, presumably because the spec is concerned with navigating over an existing tree, not how that tree came into existence. Ignorable whitespace would then be a matter for sax_builder, or whatever you use to construct a tree. (Personally, I'd prefer to lose the whitespace, and add pretty-printing to the .toxml() method.) I noticed this when parsing one of Jon Bosak's marked-up Shakespeare plays with xmllib and converting it to a DOM tree. The document node's children were [, ]. Document nodes can't have Text nodes as children [reference: DOM 1.1.1], so this would be whitespace that *must* be ignored; once the Document node does stricter checking on its children (and it will do so, after tonight), attempting to add the Text node would raise a HierarchyRequestException. Actually, this is a good side issue: what should the standard interface for making a DOM tree using a given parser be? You can use SAX easily enough: import sys from xml.sax import saxlib, saxexts from sax_builder import SaxBuilder import writer p = saxexts.make_parser() dh = SaxBuilder() p.setDocumentHandler(dh) p.parse(sys.argv[1]) doc = dh.document print doc.toxml() But SAX loses some information, such as comments, so it can't be a general solution. You'd want parser-to-DOM drivers for PyExpat, xmllib, xmlproc, etc. that preserved as much of the input XML as possible. Does anyone want to suggest an interface for this? An off-the-top-of-my-head strawman proposal might be: from xml.dom.builder import make_dom tree = make_dom(sysID or file object [, parser = 'specific parser to use'] ) -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin. -- John Von Neumann From Fred L. Drake, Jr." References: <13850.13247.104019.731837@amarok.cnri.reston.va.us> <13850.19162.682792.888605@weyr.cnri.reston.va.us> <13850.21875.949008.651441@amarok.cnri.reston.va.us> Message-ID: <13850.35315.886658.444980@weyr.cnri.reston.va.us> Andrew M. Kuchling writes: > The DOM spec doesn't actually mention ignorable whitespace, > presumably because the spec is concerned with navigating over an As we discussed, I'm remembering from an older draft of the DOM, which included an attribute called something like isIgnorableWhitespace. I still think it's a good idea, but appearantly there were problems with it. It would most certainly be useful. > But SAX loses some information, such as comments, so it can't be a > general solution. You'd want parser-to-DOM drivers for PyExpat, As does DOM, based on the same discussion. I don't have any problem with additional interfaces on the Python DOM objects that allow checking for things like ignorable white space, though, as long as reasonable behaviors are defined for operations such as reparenting. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From gstein@lyra.org Tue Oct 6 23:33:02 1998 From: gstein@lyra.org (Greg Stein) Date: Tue, 6 Oct 1998 15:33:02 -0700 Subject: [XML-SIG] DOM: Multiple proxy problem In-Reply-To: <13850.35315.886658.444980@weyr.cnri.reston.va.us> Message-ID: <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> I think it is reasonable to assume that if an application is interested in the details of white space and comments, then it can directly use a parser rather than DOM. I'd posit that most applications that are interested in the XML structure/content are interested in DOM but not in the whitespace (and other syntactic cruft). Conversely, applications that are interested in raw XML processing wouldn't be interested in the DOM representation, yet the whitespace and other cruft are relevant. YMMV. -g > -----Original Message----- > From: xml-sig-admin@python.org [mailto:xml-sig-admin@python.org]On > Behalf Of Fred L. Drake > Sent: Tuesday, October 06, 1998 2:22 PM > To: Andrew M. Kuchling > Cc: xml-sig@python.org > Subject: Re: [XML-SIG] DOM: Multiple proxy problem > > > > Andrew M. Kuchling writes: > > The DOM spec doesn't actually mention ignorable whitespace, > > presumably because the spec is concerned with navigating over an > > As we discussed, I'm remembering from an older draft of the DOM, > which included an attribute called something like > isIgnorableWhitespace. I still think it's a good idea, but > appearantly there were problems with it. It would most certainly be > useful. > > > But SAX loses some information, such as comments, so it can't be a > > general solution. You'd want parser-to-DOM drivers for PyExpat, > > As does DOM, based on the same discussion. I don't have any problem > with additional interfaces on the Python DOM objects that allow > checking for things like ignorable white space, though, as long as > reasonable behaviors are defined for operations such as reparenting. > > > -Fred > > -- > Fred L. Drake, Jr. > Corporation for National Research Initiatives > 1895 Preston White Dr. Reston, VA 20191 > > _______________________________________________ > XML-SIG maillist - XML-SIG@python.org > http://www.python.org/mailman/listinfo/xml-sig > > From akuchlin@cnri.reston.va.us Wed Oct 7 03:13:56 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Tue, 6 Oct 1998 22:13:56 -0400 (EDT) Subject: [XML-SIG] DOM: Tuesday evening's changes Message-ID: <199810070213.WAA02166@mira.erols.com> Tuesday's results: builder.py: Changed some node type constants to the _NODE form. Added Fred Drake's patch for adding a silent sanity check to builder.py html_builder.py: Tried to fix html_builder.py to work with the new DOM code. core.py: Added support for a .readonly attribute. Made DocumentType, Entity, and Notation nodes read-only. Changed the DOMException class to print more helpfully. Changed the ._parent attribute of Nodes to .parentNode. Added ._checkChild, which raises a HierarchyRequestException if the child can't be added to the requested node. (Happily, this seems to only add about 2 seconds to the 7-second parsing time of PyExpat) Use repr(node) in exception messages. Override the functions which change the children list in the Document class, to check and reset the .documentElement attribute after adding or removing a child. (Won't prevent you from adding two Element nodes until it's too late, though -- must fix that.) Fixed bug in document.getElementsByTagName(). Set .childClasses attribute, which lists the classes which can legally be a child of a given node type. Fixed some comments and docstrings. Reindented the body of the Node class. Renamed the .index method to _index sax_builder.py Only display the usage message when no arguments were provided. Also checked in two files in xmlproc that were accidentally left out of the CVS tree; sorry! -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Sappho was a worshipper of the Aphrodite cult and on the island of Lesbos there were many cliff-jumpers. They all jumped. Some may say they flew in ecstasy. If only for nine seconds -- one second for each string of the lyre. -- Peter Greenaway, _Flying Out of This World_ (1994) From Fred L. Drake, Jr." References: <13850.35315.886658.444980@weyr.cnri.reston.va.us> <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> Message-ID: <13850.57214.610455.187646@weyr.cnri.reston.va.us> Greg Stein writes: > I think it is reasonable to assume that if an application is interested in > the details of white space and comments, then it can directly use a parser > rather than DOM. I'd posit that most applications that are interested in the > XML structure/content are interested in DOM but not in the whitespace (and Frankly, what I want is to be able to address the tree structure at the application level and update things I need to, but otherwise not affecting the rest of the document instance. Given the general need to be able to handle well-formed XML in a non-destructive manner, I'd like to be able to write out the modified instance with the fewest possible changes; I'd really like "diff" output between the original and edited files to be minimal. Perhaps I'm looking for something unreasonable? I'd hope not, or I'll end up editing my bookmarks files by hand. ;-( I really don't think I'm looking for too much (note that I haven't asked anyone to implement this for me). -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From digitome@iol.ie Wed Oct 7 09:07:06 1998 From: digitome@iol.ie (Sean Mc Grath) Date: Wed, 07 Oct 1998 09:07:06 +0100 Subject: [XML-SIG] DOM: Multiple proxy problem In-Reply-To: <13850.57214.610455.187646@weyr.cnri.reston.va.us> References: <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> <13850.35315.886658.444980@weyr.cnri.reston.va.us> <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> Message-ID: <3.0.6.32.19981007090706.0096ca10@gpo.iol.ie> [Fred Drake] > Frankly, what I want is to be able to address the tree structure at >the application level and update things I need to, but otherwise not >affecting the rest of the document instance. Given the general need >to be able to handle well-formed XML in a non-destructive manner, I'd >like to be able to write out the modified instance with the fewest >possible changes; I'd really like "diff" output between the original >and edited files to be minimal. This is one of the great debate points that surrounds XML and surrounded SGML for ten years before it. In an ideal world, us programmers like to be able to do null transformations. In XML terms, we want to be able to go XML(1) -> DOM -> XML(2) such that XML(1) and XML(2) are "equivalent". the difficulty is in finding agreement on what "equivalent" means. In SGML, certain whitespace is always considered insignificant. So, if it disappears in the null transformation round-trip, no information is lost. So in SGML, if you know that white space was insignificant per your DTD you could work with clean tree structures i.e.:- This: Hello World comes out like this in a generated tree:- /---\ |foo| \---/ | |-----------| |Hello World| |-----------| In XML on the other hand, *all* whitespace is considered significant. All whitespace must be passed to the application. So at the very least you need this for null transformation purposes:- /---\ |foo| \---/ | |---------------| |\nHello World\n| |---------------| It gets a bit harier though:-(. To maintain compatability with SGML's notion of insignificant white space, validating XML parsers split white space into two types:- space known to be significant and space that can be safely thrown away. The latter is white space that occurs in content models that do not have the #PCDATA keyword. In a nutshell, all white space needs to be passed by the parser to the application - even if the application is DOM:-) Whether or not that white-space is classified into two types depends on the capabilities of the XML parser underfoot. In XML, it is up to the application (say XBEL) to make a statement about white space. "White space is always significant" or "white space after start-tags and before end-tags is always insignificant". That kind of thing. Basically, on an application by application basis the notion of "equivalence" varies. Note that I have not even mentioned the word "entities". Don't get me started on entities.... To make a long story even longer, XBEL should make some statement about the significance or otherwise of whitespace. There is no formalism for doing so, no declarative syntax. Just a statement. Sean Mc Grath def Get_URI_Of_Superlative_Scripting_Language(): return "http://www.python.org" From fleck@informatik.uni-bonn.de Wed Oct 7 13:51:26 1998 From: fleck@informatik.uni-bonn.de (Markus Fleck) Date: Wed, 07 Oct 1998 14:51:26 +0200 Subject: [XML-SIG] DOM: Multiple proxy problem References: <13850.35315.886658.444980@weyr.cnri.reston.va.us> <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> <13850.57214.610455.187646@weyr.cnri.reston.va.us> Message-ID: <361B63CE.BB5B3B66@informatik.uni-bonn.de> Fred L. Drake wrote: > Given the general need > to be able to handle well-formed XML in a non-destructive manner, I'd > like to be able to write out the modified instance with the fewest > possible changes; I'd really like "diff" output between the original > and edited files to be minimal. This must be what Tim Berners-Lee meant when he said (at WWW7): "You need to build a system that is futureproof; it's no good just making a modular system. You need to realize that your system is just going to be a module in some bigger system to come, and so you have to be part of something else, and it's a bit of a way of life." I found this to be a very nice "feature" of XML, but I also think that this implies that strict, formal DTDs shouldn't generally have the importance that they have now. Requiring a DTD for each XML document may solve the problem of "futureproof" file formats, mostly because it can serve as some kind of documentation, but it still doesn't enable a legacy XML application (using an older version of a DTD) to non-destructively process data that has been created using a more recent, but upwards-compatible version of that DTD. But then, maybe the future isn't really important. Yours, Markus. "Microsoft: The People who Brought the Y2K Bug into Software Titling" :-) -- //////////////////////////////////////////////////////////////////////////// Markus B Fleck - University of Bonn - CS Department IV - WHOIS MF5079 UNIX Administrator - comp.lang.python.announce Moderator "GNU Gather" Free Internet Groupware Project - http://cscw.net/gather/ \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ From Fred L. Drake, Jr." References: <13850.35315.886658.444980@weyr.cnri.reston.va.us> <001701bdf179$471a88a0$0e2bc0d0@suntzu.lyra.org> <13850.57214.610455.187646@weyr.cnri.reston.va.us> <361B63CE.BB5B3B66@informatik.uni-bonn.de> Message-ID: <13851.30952.700043.970182@weyr.cnri.reston.va.us> Markus Fleck writes: > This must be what Tim Berners-Lee meant when he said (at WWW7): > > "You need to build a system that is futureproof; it's no good just > making a modular system. You need to realize that your system is just > going to be a module in some bigger system to come, and so you have > to be part of something else, and it's a bit of a way of life." Nice quote. > I found this to be a very nice "feature" of XML, but I also think that > this implies that strict, formal DTDs shouldn't generally have the > importance that they have now. Requiring a DTD for each XML document Actually, I don't see how DTDs "solve" the "futureproofing" problem, and I'm not sure that "futureproofing" is well defined. (If it is, I'd be very disappointed!) There's a lot we can predict about the future, and a lot we can be wrong about. Given XML+Namespaces, it probably isn't important that any given document has exactly one document type, and the form taken by document type definitions will most likely change. However, I don't think the need for some formal notion of document types will change anytime soon (am I setting myself up for a big fall? ;). The need for specifying "intended interpretation" for document types will remain. DTDs (including the non-formal part) are one way of doing this, and offer particular benefits and limitations. They're also the best shared means of doing this at the time -- DCD and XSchema just aren't quite here yet. > may solve the problem of "futureproof" file formats, mostly because > it can serve as some kind of documentation, but it still doesn't enable > a legacy XML application (using an older version of a DTD) to > non-destructively process data that has been created using a > more recent, but upwards-compatible version of that DTD. Another concern is with well-formed XML that doesn't conform to a DTD. It is entirely reasonable to handle instances which contain completely ad-hoc tagging or elements affiliated with alternate namespaces. An application need not be "legacy" to have problems with input data when attempting to perform minimal transforms! > But then, maybe the future isn't really important. Maybe not. But I'll let my kids decide that after I die. ;-) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Wed Oct 7 15:47:19 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Wed, 7 Oct 1998 10:47:19 -0400 (EDT) Subject: [XML-SIG] RE: converting integer to string In-Reply-To: <000201bdf1a5$2a373180$fd9e2299@tim> References: <361A114A.41C6@nlr.nl> <000201bdf1a5$2a373180$fd9e2299@tim> Message-ID: <13851.31857.131427.711710@amarok.cnri.reston.va.us> Tim Peters quoted: >> Is there a function in Python to convert an integer value to >> a string? Here's an XML-based variant; the integer is marshalled into a simple format, and the resulting XML file is then parsed into a Document Object Model Tree. Obtaining the string value is then simply a matter of retrieving the single 'integer' element in the tree, and getting the value of its first child, which must be a Text node. A useful property of this implementation is that it will return the string value of the first integer in a tuple or list. Actually, given a recursive data structure, it should return the string value of the first integer found in a preorder traversal of the list. So, for example: print `xml_int2str( 5 )` print `xml_int2str( (6,5) )` print `xml_int2str( ("stupid string", 7,5) )` outputs '5' '6' '7' def xml_int2str(n): # Marshal the integer, sticking the result in a StringIO object. import StringIO mem_file = StringIO.StringIO() from xml import marshal data = marshal.dump( n, mem_file ) mem_file.seek(0) # mem_file now contains something like: # # # 5 # Parse this and convert it to a DOM tree. from xml.sax import saxexts from xml.dom.sax_builder import SaxBuilder p = saxexts.make_parser() dh = SaxBuilder() p.setDocumentHandler(dh) p.parseFile( mem_file ) # Get the root Document node. doc = dh.document # List all the 'integer' elements in the tree, and retrieve the # value of the child of the first one, which should be a Text node # containing the string '5'. integer_list = doc.getElementsByTagName( 'integer' ) integer_elem = integer_list[0] children = integer_elem.get_childNodes() return children[0].get_nodeValue() -- A.M. Kuchling http://starship.skyport.net/crew/amk/ REMIND ME AGAIN, he said, HOW THE LITTLE HORSE-SHAPED ONES MOVE. -- Death on symbolic last games, in Terry Pratchett's _Small Gods_ From Fred L. Drake, Jr." I think XML namespaces are a useful mechanism for identifying embedded fragments of XML within XML documents which conform to different DTDs, DCDs, XSchemas, or whatever is used to define document types. Use of namespaces has been proposed to embed metadata within XBEL, and I support that application. However, I just took a look at recent traffic on the xml-names-issues mailing list (http://lists.w3.org/Archives/Public/xml-names-issues/), and things don't at all look like they're settling down, or at least that they shouldn't. Does anyone here have an "inside scoop" on the current state of the draft? I've read the 1998-09-19 draft roughly, and it leaves me with a number of questions. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Thu Oct 8 02:13:38 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Wed, 7 Oct 1998 21:13:38 -0400 Subject: [XML-SIG] Fate of pydom.py? Message-ID: <199810080113.VAA00886@207-172-50-73.s73.tnt15.ann.erols.com> pydom.py is a little module that forms part of the DOM package; the docstring describes it as "DOM element construction using Python syntax.". It defines a subclass of the Element class which adds a bit of syntactic sugar to element creation. It accepts a variable number of regular and keyword arguments. Dictionaries are treated as containing name,attribute pairs, as are keyword arguments. Tuples and lists are assumed to contain other nodes which will become children of the new Element node; strings are converted to Text nodes and added as children. An example will help. To create this pseudo-HTML XML:

Heading

I'm the first paragraph. You could do this: h1 = Element('H1', 'Heading') body = Element(h1, "I'm the first paragraph", {'background':'#ffffff'}, textcolor = "#000000") There's also the start of some operator overloading; the Element class has an __lshift__ function which inserts the right-hand argument at the front of the children, so parent << child will add the child to the parent node. Question: What should we do with pydom.py? My inclination is to drop it. I don't mind adding some syntactic sugar to Element creation, though I'd like to simplify it; perhaps just having keywords added as attributes, so Element('body', background='#ffffff', textcolor = '#000000') would work. I'm less sure about the operator overloading idea, and would prefer to drop it. There doesn't seem to be a natural mapping between tree operations and Python's operators. Using << and >> for appending at either end seems like a bit of a stretch, for example. Also, it might sometimes be confusing; if + is used for append, for example, people might expect TextNode + "foo" to change the string value of TextNode. If no one voices any opinions on this, I'll drop pydom.py, and add the keyword shortcut to Document.createElement(). -- A.M. Kuchling http://starship.skyport.net/crew/amk/ I've always believed that guilt was an emotion for weaker men -- so where does that leave me? -- Wesley Dodds, in SANDMAN MYSTERY THEATRE: "The Cannon", act I From Jeff.Johnson@stn.siemens.com Thu Oct 8 18:42:32 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Thu, 8 Oct 1998 13:42:32 -0400 Subject: [XML-SIG] pydom Message-ID: <85256697.00613715.00@BI01.boca.ssc.siemens.com> I use pyhtml.py, which uses pydom, to create my HTML elements because it saves lots of lines of code. It may not be quite so readable but I don't think it's that bad. In fact, I think it's more readable than having to read the 10 lines of code resulting from using core.py. I also miss the ability to specify the attribute names and values in the core createElement(). Consider the following using pyhtml.py: html = HTML(HEAD(TITLE(self.title)),BODY(list,BACKGROUND=self.background)) Now using core.py (maybe there is a much easier way to do this with core.py?): html = doc.createElement('HTML') head = doc.createElement('HEAD') title = doc.createElement('TITLE') title.appendChild(doc.createTextNode(titleText)) body = doc.createElement('BODY') body.setAttribute('BACKGROUND',background) html.appendChild(head) html.appendChild(body) head.appendChild(title) body.appendChild(list) From Fred L. Drake, Jr." I have a draft copy of the XBEL documentation. It's available at: http://www.python.org/sigs/xml-sig/xbel/doc/ It is incomplete, but I am interested in comments. I've attached the current DTD below; the only change is that now requires at least one . Software that removes the last should discard the as well. I don't recall that we reached resolution on the metadata stuff; I'll leave it alone unless issues are brought up at this point. We also haven't reached a conclusion regarding the use of parameter entities to make the DTD more easily extended; are there any remaining issues there? My inclination is to "do it right" to allow DTD reuse, but there were sentiments that went the other way. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From gstein@lyra.org Thu Oct 8 23:15:37 1998 From: gstein@lyra.org (Greg Stein) Date: Thu, 08 Oct 1998 15:15:37 -0700 Subject: [XML-SIG] XBEL documentation, draft References: <13853.13854.985841.118112@weyr.cnri.reston.va.us> Message-ID: <361D3989.16D0856E@lyra.org> Fred L. Drake wrote: > > I have a draft copy of the XBEL documentation. It's available at: > > http://www.python.org/sigs/xml-sig/xbel/doc/ > > It is incomplete, but I am interested in comments. I've attached > the current DTD below; the only change is that now requires at > least one . Software that removes the last > should discard the as well. > I don't recall that we reached resolution on the metadata stuff; > I'll leave it alone unless issues are brought up at this point. > We also haven't reached a conclusion regarding the use of parameter > entities to make the DTD more easily extended; are there any remaining > issues there? My inclination is to "do it right" to allow DTD reuse, > but there were sentiments that went the other way. The DTD wasn't attached :-) Nevertheless, I feel that the metadata element has very little purpose. Given the presence of XML Namespaces (http://www.w3.org/TR/WD-xml-names), I don't think that the metadata tag adds any value. Rather than use , we can use . Of course, there are other ways to specify the use of a namespace, but the point is that a "metadata-like" facility has already been defined in a standardized fashion (with the caveat of minor perturbations to the working draft prior to IETF acceptance). still adds some value, in that the spec can say "app-specific extensions should go into the element; any application should retain all child elements of the element when manipulating items in the XBEL document." %common.attrs% is not very useful as an extension. It is located to broadly for somebody to extend the DTD. If you try to add something into %common.attr% it shows up EVERYWHERE. eek. Personally, I'm of the mind that somebody can issue a second version of the DTD, rather than attempt to extend it. At the present time, I'm not aware of apps that actually make use of extending a DTD. Why should we make an attempt which might not even be in the right direction? I'd say: let some precedent exist before biting this off. Further, given the presence of XML namespaces, an XBEL document can simply be incorporated *within* another XML document. i.e. extension via containment, rather than extension via derivation. The simplest designs always seem to be the most used designs, in my book. :-) (and I'm a big fan of KISS) Cheers, -g -- Greg Stein (gstein@lyra.org) From gstein@lyra.org Fri Oct 9 07:55:17 1998 From: gstein@lyra.org (Greg Stein) Date: Thu, 08 Oct 1998 23:55:17 -0700 Subject: [XML-SIG] pydom References: <85256697.00613715.00@BI01.boca.ssc.siemens.com> Message-ID: <361DB355.6D572FAA@lyra.org> This is a multi-part message in MIME format. --------------77A2CB87277BD0D1105FC50E Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Jeff.Johnson@stn.siemens.com wrote: > > I use pyhtml.py, which uses pydom, to create my HTML elements because it > saves lots of lines of code. It may not be quite so readable but I don't > think it's that bad. In fact, I think it's more readable than having to > read the 10 lines of code resulting from using core.py. I also miss the > ability to specify the attribute names and values in the core > createElement(). > > Consider the following using pyhtml.py: > > html = > HTML(HEAD(TITLE(self.title)),BODY(list,BACKGROUND=self.background)) > > Now using core.py (maybe there is a much easier way to do this with > core.py?): > ... lengthy example elided ... I saw this and got to thinking about some text (XML/HTML) generation. I looked at pyhtml.py and saw that it only encoded a specific subset of available tags. "There has got to be a clean way to have an open-ended system," I thought :-) [ I'm not familiar with HTMLgen, but I'm copying Robin in case he finds this interesting... (and his xml-sig delivery is disabled for some reason) ] Anyhow, I've attached a quick pass at some code to do (XML) generation. The above example would be written like this: import xmlgen f = xmlgen.Factory() html = f.html[f.head.title(self.title), f.body(background=self.background)[list]] The text is retrieved using str(html). There are two ways to create an element: - via the factory "f". f.foo creates and returns a element - via another element. elem.foo creates, and inserts into a self, a element Given an element, you can insert 0 or more items of CDATA into it: elem('piece 1', 'piece 2') Usually used as: elem.subelem('cdata') or as: f.elem('cdata') Given an element, you can insert attributes into it: elem(attr1='value 1', attr2='value 2') Again, usually used when creating them: elem.subelem(attr1='value1') Given a number of elements, you can insert 1 or more elements as children of another using index notation: elem1[elem2, elem3, elem4] Finally, when creating a child element, the return value can create a nested child element: elem.subelem.subsubelem In this last form, the return value is tricky. If you str() it, then you get "elem" and its children. Otherwise, it acts like the "subsubelem" -- elem creation, insertion, and CDATA/attribute will operate on subsubelem. This allows you to do things like: body = f.body(bgcolor='#ffffff').p.a(href='foo.html').img(src='image.gif') html = f.html[f.head.title('title'), body] Some relatively simple rules, some funky code, and hopefully a clear result for assembling text. Comments welcome :-) Cheers, -g -- Greg Stein (gstein@lyra.org) --------------77A2CB87277BD0D1105FC50E Content-Type: text/plain; charset=us-ascii; name="xmlgen.py" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="xmlgen.py" # # XML generation module # import string class Element: def __init__(self, name, *cdata, **attrs): self.__name = name self.__children = [] self.__cdata = cdata self.__attrs = attrs def __getattr__(self, name): if name[:2] == '__': raise AttributeError, name child = Element(name) self.__children.append(child) return Intermediate(self, child) def __call__(self, *cdata, **attrs): self.__cdata = self.__cdata + cdata self.__attrs.update(attrs) return self def __getitem__(self, items): if type(items) == type(()) or type(items) == type([]): self.__children = self.__children + list(items) else: self.__children.append(items) return self def __str__(self): s = '<' + self.__name for name, value in self.__attrs.items(): s = s + ' ' + name + '="' + str(value) + '"' if self.__cdata or self.__children: s = s + '>' + string.joinfields(self.__cdata, '') for child in self.__children: s = s + str(child) s = s + '' else: s = s + '/>' return s class Factory: def __getattr__(self, name): if name[:2] == '__': raise AttributeError, name return Element(name) class Intermediate: def __init__(self, parent, child): self.__parent = parent self.__child = child def __getattr__(self, name): inter = getattr(self.__child, name) return Intermediate(self.__parent, inter.__child) def __call__(self, *cdata, **attrs): apply(self.__child, cdata, attrs) return self def __getitem__(self, items): self.__child[items] return self def __str__(self): return str(self.__parent) def test(): f = Factory() l = f.ul[f.li('list 1'), f.li('list 2'), f.li('list 3')] l[f.li('list 4')] top = f.html[f.head.title.i('title'), f.body(bgcolor='#ffffff')[f.h1('heading'),f.p('text'),l]] print top --------------77A2CB87277BD0D1105FC50E-- From akuchlin@cnri.reston.va.us Fri Oct 9 15:57:29 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Fri, 9 Oct 1998 10:57:29 -0400 (EDT) Subject: [XML-SIG] pydom In-Reply-To: <85256697.00613715.00@BI01.boca.ssc.siemens.com> References: <85256697.00613715.00@BI01.boca.ssc.siemens.com> Message-ID: <13854.9127.940596.155564@amarok.cnri.reston.va.us> Jeff.Johnson@stn.siemens.com writes: >Consider the following using pyhtml.py: > > html = >HTML(HEAD(TITLE(self.title)),BODY(list,BACKGROUND=self.background)) > The builder.py module may help; using it, the code would look something like: from xml.dom import builder b=builder.Builder() b.startElement('HTML', {}) b.startElement('HEAD', {}) b.startElement('TITLE', {}) 'print b.document.toxml()' outputs: '\012\012</HEAD></HTML>' b.text("title Text") print b.document.toxml() '<?xml version="1.0"?>\012<!DOCTYPE XXX>\012<HTML> <HEAD><TITLE>title Text' b.pop() b.pop() b.startElement('BODY', {'BACKGROUND':'#00ffff'}) print b.document.toxml() '\012\012 title Text ' -- A.M. Kuchling http://starship.skyport.net/crew/amk/ The Social Sciences are good at accounting for disasters once they have taken place. -- Claude T. Bissell From Jeff.Johnson@stn.siemens.com Fri Oct 9 19:12:33 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Fri, 9 Oct 1998 14:12:33 -0400 Subject: [XML-SIG] DOM circular refs Message-ID: <85256698.0063FD3B.00@BI01.boca.ssc.siemens.com> --0__=A2kFZT3qDsXrOyFApQpGBoEJxCoVbsRUYNc21gz18D7Arav2w7NYsuZs Content-type: text/plain; charset=us-ascii Content-Disposition: inline Hi Andrew, I think I fixed the last of the circular references but to do it I had to make the Document.documentElement a function called get_documentElement() which generates a new Element at call time. I've got a small test file and a new version or core.py that I temporarily call coretest.py. According to the DOM spec, the documentElement is a readonly, convenience attribute. By making it a function, we can enforce the readonly bit. I patched a few other things too. I've improved my test suite too. I left the code that checks __init__ vs __del__, otherwise my test code can't work, it should be taken out of the released code... (See attached file: coretest.py)(See attached file: test.py) --0__=A2kFZT3qDsXrOyFApQpGBoEJxCoVbsRUYNc21gz18D7Arav2w7NYsuZs Content-type: application/octet-stream; name="coretest.py" Content-Disposition: attachment; filename="coretest.py" Content-transfer-encoding: base64 JycnDQpjb3JlLnB5OiBgbGlnaHQnIGltcGxlbWVudGF0aW9uIG9mIHRoZSBEb2N1bWVudCBPYmpl Y3QgTW9kZWwgKGNvcmUpIGxldmVsIDEuDQoNClJlZmVyZW5jZTogaHR0cDovL3d3dy53My5vcmcv VFIvV0QtRE9NL2xldmVsLW9uZS1jb3JlDQoNCkRldmlhdGlvbnMgZnJvbSB0aGUgc3BlY3M6DQoN Ci0tIHRoZXJlIGlzIGN1cnJlbnRseSBubyBEb2N1bWVudEZyYWdtZW50Lg0KDQpTbywgdXNlZnVs IGNsYXNzZXMgaW4gdGhpcyBtb2R1bGUgYXJlIE5vZGUgKGFic3RyYWN0KSBhbmQgaXRzDQooY29u Y3JldGUpIHN1YmNsYXNzZXMgLS0gRG9jdW1lbnQsIEVsZW1lbnQsIFRleHQsIENvbW1lbnQsDQpQ cm9jZXNzaW5nSW5zdHJ1Y3Rpb24gLS0gYWxsIG9mIHdoaWNoIHNob3VsZCBiZSBpbnN0YW50aWF0 ZWQgdGhvdWdoDQp0aGUgcmVsZXZhbnQgY3JlYXRlWFhYKCkgbWV0aG9kcyBvbiBhIERvY3VtZW50 IG9iamVjdC4gIA0KDQpUeXBpY2FsIHVzYWdlIFhYWA0KDQpmcm9tIHhtbC5kb20uY29yZSBpbXBv cnQgKg0KDQpYWFggZXhhbXBsZSBoZXJlDQouLi4NCg0KJycnDQoNCmltcG9ydCBzdHJpbmcNCmZy b20geG1sLnV0aWxzIGltcG9ydCBlc2NhcGUNCg0KIyBSZWZlcmVuY2VzIGluc2lkZSBzcXVhcmUg YnJhY2tldHMsIHN1Y2ggYXMgWzEuNV0sIGFyZSB0byBhIHNlY3Rpb24gaW4gdGhlDQojIE9jdG9i ZXIgMXN0IERPTSBMZXZlbCBPbmUgc3BlY2lmaWNhdGlvbi4NCg0KIw0KIyBNb2R1bGUtbGV2ZWwg ZGVmaW5pdGlvbnMNCiMNCg0KIyBFeGNlcHRpb24gY29kZXMgWzEuMl0NCklOREVYX1NJWkVfRVJS ICAgICAgICAgICAgICA9IDENCkRPTVNUUklOR19TSVpFX0VSUiAgICAgICAgICA9IDINCkhJRVJB UkNIWV9SRVFVRVNUX0VSUiAgICAgICA9IDMNCldST05HX0RPQ1VNRU5UX0VSUiAgICAgICAgICA9 IDQNCklOVkFMSURfQ0hBUkFDVEVSX0VSUiAgICAgICA9IDUNCk5PX0RBVEFfQUxMT1dFRF9FUlIg ICAgICAgICA9IDYNCk5PX01PRElGSUNBVElPTl9BTExPV0VEX0VSUiA9IDcNCk5PVF9GT1VORF9F UlIgICAgICAgICAgICAgICA9IDgNCk5PVF9TVVBQT1JURURfRVJSICAgICAgICAgICA9IDkNCklO VVNFX0FUVFJJQlVURV9FUlIgICAgICAgICA9IDEwDQoNCiMgRXhjZXB0aW9ucyAobm93IGNoYW5n ZWQgdG8gY2xhc3MtYmFzZWQuIC0tYW1rKQ0KDQpjbGFzcyBET01FeGNlcHRpb246DQogICAgZGVm IF9faW5pdF9fKHNlbGYsIG1zZyk6DQogICAgICAgIHNlbGYubXNnID0gbXNnDQogICAgZGVmIF9f cmVwcl9fKHNlbGYpOg0KICAgICAgICByZXR1cm4gc2VsZi5tc2cNCiAgICANCmNsYXNzIEluZGV4 U2l6ZUV4Y2VwdGlvbihET01FeGNlcHRpb24pOg0KICAgIGNvZGUgPSBJTkRFWF9TSVpFX0VSUg0K Y2xhc3MgRE9NU3RyaW5nU2l6ZUV4Y2VwdGlvbihET01FeGNlcHRpb24pOg0KICAgIGNvZGUgPSBE T01TVFJJTkdfU0laRV9FUlINCmNsYXNzIEhpZXJhcmNoeVJlcXVlc3RFeGNlcHRpb24oRE9NRXhj ZXB0aW9uKToNCiAgICBjb2RlID0gSElFUkFSQ0hZX1JFUVVFU1RfRVJSDQpjbGFzcyBXcm9uZ0Rv Y3VtZW50RXhjZXB0aW9uKERPTUV4Y2VwdGlvbik6DQogICAgY29kZSA9IFdST05HX0RPQ1VNRU5U X0VSUg0KY2xhc3MgTm9EYXRhQWxsb3dlZEV4Y2VwdGlvbihET01FeGNlcHRpb24pOg0KICAgIGNv ZGUgPSBOT19EQVRBX0FMTE9XRURfRVJSDQpjbGFzcyBOb01vZGlmaWNhdGlvbkFsbG93ZWRFeGNl cHRpb24oRE9NRXhjZXB0aW9uKToNCiAgICBjb2RlID0gTk9fTU9ESUZJQ0FUSU9OX0FMTE9XRURf RVJSDQpjbGFzcyBOb3RGb3VuZEV4Y2VwdGlvbihET01FeGNlcHRpb24pOg0KICAgIGNvZGUgPSBO T1RfRk9VTkRfRVJSDQpjbGFzcyBOb3RTdXBwb3J0ZWRFeGNlcHRpb24oRE9NRXhjZXB0aW9uKToN CiAgICBjb2RlID0gTk9UX1NVUFBPUlRFRF9FUlINCmNsYXNzIEluVXNlQXR0cmlidXRlRXhjZXB0 aW9uOg0KICAgIGNvZGUgPSBJTlVTRV9BVFRSSUJVVEVfRVJSDQoNCiMgT2xkIGV4Y2VwdGlvbnMg KGRlcHJlY2F0ZWQpDQpjbGFzcyBOb1N1Y2hOb2RlRXhjZXB0aW9uKERPTUV4Y2VwdGlvbik6IHBh c3MNCmNsYXNzIE5vdE15Q2hpbGRFeGNlcHRpb24oRE9NRXhjZXB0aW9uKTogcGFzcw0KY2xhc3Mg Tm90SW1wbGVtZW50ZWRFeGNlcHRpb24oRE9NRXhjZXB0aW9uKTogcGFzcw0KDQojIE5vZGUgdHlw ZXMuIA0KDQpFTEVNRU5UICAgICAgICAgICAgICAgID0gRUxFTUVOVF9OT0RFICAgICAgICAgICAg ICAgID0gMQ0KQVRUUklCVVRFICAgICAgICAgICAgICA9IEFUVFJJQlVURV9OT0RFICAgICAgICAg ICAgICA9IDINClRFWFQgICAgICAgICAgICAgICAgICAgPSBURVhUX05PREUgICAgICAgICAgICAg ICAgICAgPSAzDQpDREFUQV9TRUNUSU9OICAgICAgICAgID0gQ0RBVEFfU0VDVElPTl9OT0RFICAg ICAgICAgID0gNA0KRU5USVRZX1JFRkVSRU5DRSAgICAgICA9IEVOVElUWV9SRUZFUkVOQ0VfTk9E RSAgICAgICA9IDUNCkVOVElUWSAgICAgICAgICAgICAgICAgPSBFTlRJVFlfTk9ERSAgICAgICAg ICAgICAgICAgPSA2DQpQUk9DRVNTSU5HX0lOU1RSVUNUSU9OID0gUFJPQ0VTU0lOR19JTlNUUlVD VElPTl9OT0RFID0gNw0KQ09NTUVOVCAgICAgICAgICAgICAgICA9IENPTU1FTlRfTk9ERSAgICAg ICAgICAgICAgICA9IDgNCkRPQ1VNRU5UICAgICAgICAgICAgICAgPSBET0NVTUVOVF9OT0RFICAg ICAgICAgICAgICAgPSA5DQpET0NVTUVOVF9UWVBFICAgICAgICAgID0gRE9DVU1FTlRfVFlQRV9O T0RFICAgICAgICAgID0gMTANCkRPQ1VNRU5UX0ZSQUdNRU5UICAgICAgPSBET0NVTUVOVF9GUkFH TUVOVF9OT0RFICAgICAgPSAxMQ0KTk9UQVRJT04gICAgICAgICAgICAgICA9IE5PVEFUSU9OX05P REUgICAgICAgICAgICAgICA9IDEyDQoNCg0KZGVmIGhhc0ZlYXR1cmUoZmVhdHVyZSwgdmVyc2lv biA9IE5vbmUpOg0KICAgICIiIlRlc3QgaWYgdGhlIERPTSBpbXBsZW1lbnRhdGlvbiBpbXBsZW1l bnRzIGEgc3BlY2lmaWMgZmVhdHVyZS4NCiAgICBmZWF0dXJlIC0tIHBhY2thZ2UgbmFtZSBvZiB0 aGUgZmVhdHVyZSB0byB0ZXN0LiAgSW4gTGV2ZWwgMSBET00gdGhlDQogICAgICAgICAgICAgICBs ZWdhbCB2YWx1ZXMgYXJlICdIVE1MJyBhbmQgJ1hNTCcuDQogICAgdmVyc2lvbiAtLSB2ZXJzaW9u IG51bWJlciBvZiB0aGUgcGFja2FnZSB0byB0ZXN0IChvcHRpb25hbCkNCiAgICAiIiINCiAgICBm ZWF0dXJlPXN0cmluZy5sb3dlcihmZWF0dXJlKQ0KICAgIA0KICAgIGlmIGZlYXR1cmUgPT0gJ2h0 bWwnOiByZXR1cm4gMA0KICAgIGlmIGZlYXR1cmUgPT0gJ3htbCc6DQogICAgICAgIGlmIHZlcnNp b24gaXMgTm9uZTogcmV0dXJuIDENCiAgICAgICAgaWYgdmVyc2lvbiA9PSAnMS4wJzogcmV0dXJu IDENCiAgICAgICAgcmV0dXJuIDANCg0KZGVmIGNyZWF0ZURvY3VtZW50KCk6DQogICAgZCA9IF9u b2RlRGF0YShET0NVTUVOVF9OT0RFKQ0KICAgIGQubmFtZSA9ICcjZG9jdW1lbnQnDQogICAgZC52 YWx1ZSA9IGQuYXR0cmlidXRlcyA9IE5vbmUNCiAgICBkID0gRG9jdW1lbnQoZCwgTm9uZSwgTm9u ZSkNCiAgICAjZC5fZG9jdW1lbnQgPSBkDQogICAgcmV0dXJuIGQNCg0KaW1wb3J0IFVzZXJMaXN0 LCBVc2VyRGljdA0KDQpjbGFzcyBOb2RlTGlzdChVc2VyTGlzdC5Vc2VyTGlzdCk6DQogICAgIiIi QW4gb3JkZXJlZCBjb2xsZWN0aW9uIG9mIG5vZGVzLCBlcXVpdmFsZW50IHRvIGEgUHl0aG9uIGxp c3QuICBUaGUgb25seQ0KICAgIGRpZmZlcmVuY2UgaXMgdGhhdCBhbiAuaXRlbSgpIG1ldGhvZCBh bmQgYSAubGVuZ3RoIGF0dHJpYnV0ZSBhcmUgYWRkZWQuDQogICAgIiIiDQogICAgaXRlbSA9IFVz ZXJMaXN0LlVzZXJMaXN0Ll9fZ2V0aXRlbV9fDQogICAgZ2V0X2xlbmd0aCA9IFVzZXJMaXN0LlVz ZXJMaXN0Ll9fbGVuX18NCg0KY2xhc3MgTmFtZWROb2RlTWFwKFVzZXJEaWN0LlVzZXJEaWN0KToN CiAgICAiIiJVc2VkIHRvIHJlcHJlc2VudCBhIGNvbGxlY3Rpb24gb2Ygbm9kZXMgdGhhdCBjYW4g YmUgYWNjZXNzZWQgYnkgbmFtZS4NCiAgICBFcXVpdmFsZW50IHRvIGEgUHl0aG9uIGRpY3Rpb25h cnksIHdpdGggdmFyaW91cyBhbGlhc2VzIGFkZGVkIHN1Y2ggYXMNCiAgICBnZXROYW1lZEl0ZW0g YW5kIHJlbW92ZU5hbWVkSXRlbS4NCiAgICAiIiINCg0KICAgIGdldE5hbWVkSXRlbSA9IFVzZXJE aWN0LlVzZXJEaWN0Ll9fZ2V0aXRlbV9fDQogICAgcmVtb3ZlTmFtZWRJdGVtID0gVXNlckRpY3Qu VXNlckRpY3QuX19kZWxpdGVtX18NCiAgICBnZXRfbGVuZ3RoID0gVXNlckRpY3QuVXNlckRpY3Qu X19sZW5fXw0KICAgIGRlZiBzZXROYW1lZEl0ZW0oc2VsZiwgYXJnKToNCiAgICAgICAga2V5ID0g YXJnLm5vZGVOYW1lDQogICAgICAgIHNlbGYuZGF0YVtrZXldID0gYXJnDQoNCiAgICBkZWYgaXRl bShzZWxmLCBpbmRleCk6DQogICAgICAgIHJldHVybiBzZWxmLmRhdGEudmFsdWVzWyBpbmRleCBd DQoNCmNsYXNzIF9ub2RlRGF0YToNCiAgICAiIiJDbGFzcyB1c2VkIGZvciBzdG9yaW5nIHRoZSBk YXRhIGZvciBhIHNpbmdsZSBub2RlLiAgSW5zdGFuY2VzIG9mDQogICAgdGhpcyBjbGFzcyBzaG91 bGQgbmV2ZXIgYmUgcmV0dXJuZWQgdG8gdXNlcnMgb2YgdGhlIERPTSBpbXBsZW1lbnRhdGlvbi4i IiINCiAgICBOb2RlX2NvdW50ZXIgPSAwDQogICAgZGVmIF9faW5pdF9fKHNlbGYsIHR5cGUpOg0K ICAgICAgICBzZWxmLnR5cGUgPSB0eXBlDQogICAgICAgIHNlbGYuY2hpbGRyZW4gPSBbXQ0KICAg ICAgICBzZWxmLm5hbWUgPSBzZWxmLnZhbHVlID0gc2VsZi5hdHRyaWJ1dGVzID0gTm9uZQ0KICAg ICAgICBfbm9kZURhdGEuTm9kZV9jb3VudGVyID0gX25vZGVEYXRhLk5vZGVfY291bnRlciArIDEN CiAgICAgICAgcHJpbnQgJ19ub2RlRGF0YVx0aW5pdFx0JXNcdCVzJyAlIChfbm9kZURhdGEuTm9k ZV9jb3VudGVyLCBzZWxmLm5hbWUpDQoNCiAgICBkZWYgX19kZWxfXyhzZWxmKToNCiAgICAgICAg X25vZGVEYXRhLk5vZGVfY291bnRlciA9IF9ub2RlRGF0YS5Ob2RlX2NvdW50ZXIgLTENCiAgICAg ICAgcHJpbnQgJ19ub2RlRGF0YVx0ZGVsXHQlc1x0JXMnICUgKF9ub2RlRGF0YS5Ob2RlX2NvdW50 ZXIsIHNlbGYubmFtZSkNCg0KDQpjbGFzcyBOb2RlOg0KICAgICcnJ0Jhc2UgY2xhc3MgZm9yIGdy b3ZlIG5vZGVzIGluIERPTSBtb2RlbC4gIFByb3hpZXMgYW4gaW5zdGFuY2UNCiAgICBvZiB0aGUg X25vZGVEYXRhIGNsYXNzLicnJw0KDQoNCiAgICByZWFkb25seSA9IDANCiAgICBOb2RlX2NvdW50 ZXIgPSAwDQogICAgZGVmIF9faW5pdF9fKHNlbGYsIG5vZGUsIHBhcmVudCA9IE5vbmUsIGRvY3Vt ZW50ID0gTm9uZSk6DQogICAgICAgIHNlbGYuX25vZGUgPSBub2RlDQogICAgICAgIHNlbGYucGFy ZW50Tm9kZSA9IHBhcmVudA0KICAgICAgICBzZWxmLl9kb2N1bWVudCA9IGRvY3VtZW50DQogICAg ICAgIE5vZGUuTm9kZV9jb3VudGVyID0gTm9kZS5Ob2RlX2NvdW50ZXIgKyAxDQogICAgICAgICNh c3NlcnQgTm9kZS5Ob2RlX2NvdW50ZXIgPCAzDQogICAgICAgIHByaW50ICdOb2RlICAgICAgXHRp bml0XHQlc1x0JXMnICUgKE5vZGUuTm9kZV9jb3VudGVyLCBzZWxmLmdldF9ub2RlTmFtZSgpKQ0K DQogICAgZGVmIF9fZGVsX18oc2VsZik6DQogICAgICAgIE5vZGUuTm9kZV9jb3VudGVyID0gTm9k ZS5Ob2RlX2NvdW50ZXIgLTENCiAgICAgICAgcHJpbnQgJ05vZGUgICAgICBcdGRlbFx0JXNcdCVz JyAlIChOb2RlLk5vZGVfY291bnRlciwgc2VsZi5nZXRfbm9kZU5hbWUoKSkNCg0KICAgIGRlZiBf aW5kZXgoc2VsZik6DQogICAgICAgICJSZXR1cm4gdGhlIGluZGV4IG9mIHRoaXMgY2hpbGQgaW4g aXRzIHBhcmVudCdzIGNoaWxkIGxpc3QiDQogICAgICAgIGlmIHNlbGYucGFyZW50Tm9kZToNCiAg ICAgICAgICAgIHJldHVybiBzZWxmLnBhcmVudE5vZGUuX25vZGUuY2hpbGRyZW4uX2luZGV4KHNl bGYpDQogICAgICAgIGVsc2U6DQogICAgICAgICAgICByZXR1cm4gLTENCg0KICAgIGRlZiBfY2hl Y2tDaGlsZChzZWxmLCBjaGlsZCwgcGFyZW50KToNCiAgICAgICAgaWYgcGFyZW50Ll9ub2RlLnR5 cGUgPT0gRE9DVU1FTlQ6DQogICAgICAgICAgICBpZiBjaGlsZC5fZG9jdW1lbnQgIT0gcGFyZW50 Og0KICAgICAgICAgICAgICAgIHJhaXNlIFdyb25nRG9jdW1lbnRFeGNlcHRpb24oImNoaWxkIGNy ZWF0ZWQgZnJvbSBhIGRpZmZlcmVudCBkb2N1bWVudCIpDQogICAgICAgIGVsaWYgY2hpbGQuX2Rv Y3VtZW50ICE9IHBhcmVudC5fZG9jdW1lbnQ6DQogICAgICAgICAgICByYWlzZSBXcm9uZ0RvY3Vt ZW50RXhjZXB0aW9uKCJjaGlsZCBjcmVhdGVkIGZyb20gYSBkaWZmZXJlbnQgZG9jdW1lbnQiKQ0K DQogICAgICAgICJSYWlzZSBIaWVyYXJjaHlSZXF1ZXN0RXhjZXB0aW9uIGlmIHRoZSBjaGlsZCBj YW4ndCBiZSBhZGRlZCINCiAgICAgICAgZm9yIGtsYXNzIGluIHNlbGYuY2hpbGRDbGFzc2VzOg0K ICAgICAgICAgICAgaWYgaXNpbnN0YW5jZShjaGlsZCwga2xhc3MpOiBicmVhaw0KICAgICAgICBl bHNlOg0KICAgICAgICAgICAgcmFpc2UgSGllcmFyY2h5UmVxdWVzdEV4Y2VwdGlvbiwgXA0KICAg ICAgICAgICAgICAgICAgIiVzIGNhbm5vdCBiZSBjaGlsZCBvZiAlcyIgJSAocmVwcihjaGlsZCks IHJlcHIoc2VsZikgKQ0KDQogICAgICAgIGNuID0gY2hpbGQuX25vZGUgOyBwPXNlbGYNCiAgICAg ICAgd2hpbGUgcCBpcyBub3QgTm9uZToNCiAgICAgICAgICAgIGlmIHAuX25vZGUgaXMgY246IA0K ICAgICAgICAgICAgICAgIHJhaXNlIEhpZXJhcmNoeVJlcXVlc3RFeGNlcHRpb24sIFwNCiAgICAg ICAgICAgICAgICAgICAgICAiJXMgaXMgYW4gYW5jZXN0b3Igb2YgJXMiICUgKHJlcHIoY2hpbGQp LCByZXByKHBhcmVudCkgKQ0KICAgICAgICAgICAgcCA9IHAucGFyZW50Tm9kZQ0KICAgICAgICAN CiAgICAjIGdldC9zZXQgYXR0cmlidXRlcw0KDQogICAgZGVmIGdldF9ub2RlTmFtZShzZWxmKToN CiAgICAgICAgcmV0dXJuIHNlbGYuX25vZGUubmFtZQ0KICAgIGdldF9uYW1lID0gZ2V0X25vZGVO YW1lDQoNCiAgICBkZWYgZ2V0X25vZGVWYWx1ZShzZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYu X25vZGUudmFsdWUNCiAgICBnZXRfdmFsdWUgPSBnZXRfbm9kZVZhbHVlDQoNCiAgICBkZWYgZ2V0 X25vZGVUeXBlKHNlbGYpOg0KICAgICAgICByZXR1cm4gc2VsZi5fbm9kZS50eXBlDQoNCiAgICBk ZWYgZ2V0X3BhcmVudE5vZGUoc2VsZik6DQogICAgICAgIHJldHVybiBzZWxmLnBhcmVudE5vZGUN Cg0KICAgIGRlZiBnZXRfY2hpbGROb2RlcyhzZWxmKToNCiAgICAgICAgTCA9IHNlbGYuX25vZGUu Y2hpbGRyZW5bOl0NCiAgICAgICAgIyBDb252ZXJ0IHRoZSBsaXN0IG9mIF9ub2RlRGF0YSBpbnN0 YW5jZXMgaW50byBhIGxpc3Qgb2YNCiAgICAgICAgIyB0aGUgYXBwcm9wcmlhdGUgTm9kZSBzdWJj bGFzc2VzDQogICAgICAgIGZvciBpIGluIHJhbmdlKGxlbihMKSk6DQogICAgICAgICAgICBMW2ld ID0gIE5PREVfQ0xBU1NbIExbaV0udHlwZSBdIChMW2ldLCBzZWxmLCBzZWxmLl9kb2N1bWVudCkN CiAgICAgICAgcmV0dXJuIE5vZGVMaXN0KEwpDQoNCiAgICBkZWYgZ2V0X2ZpcnN0Q2hpbGQoc2Vs Zik6DQogICAgICAgIGlmIHNlbGYuX25vZGUuY2hpbGRyZW46DQogICAgICAgICAgICBuID0gc2Vs Zi5fbm9kZS5jaGlsZHJlblswXQ0KICAgICAgICAgICAgcmV0dXJuIE5PREVfQ0xBU1NbIG4udHlw ZSBdIChuLCBzZWxmLCBzZWxmLl9kb2N1bWVudCkNCiAgICAgICAgZWxzZToNCiAgICAgICAgICAg IHJldHVybiBOb25lDQoNCiAgICBkZWYgZ2V0X2xhc3RDaGlsZChzZWxmKToNCiAgICAgICAgaWYg c2VsZi5fbm9kZS5jaGlsZHJlbjoNCiAgICAgICAgICAgIG4gPSBzZWxmLl9ub2RlLmNoaWxkcmVu Wy0xXQ0KICAgICAgICAgICAgcmV0dXJuIE5PREVfQ0xBU1NbIG4udHlwZSBdIChuLCBzZWxmLCBz ZWxmLl9kb2N1bWVudCkNCiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIHJldHVybiBOb25lDQoN CiAgICBkZWYgZ2V0X3ByZXZpb3VzU2libGluZyhzZWxmKToNCiAgICAgICAgaWYgc2VsZi5wYXJl bnROb2RlIGlzIE5vbmU6IHJldHVybiBOb25lDQogICAgICAgIGkgPSBzZWxmLl9pbmRleCgpDQog ICAgICAgIGlmIGkgPD0gMDoNCiAgICAgICAgICAgIHJldHVybiBOb25lDQogICAgICAgIGVsc2U6 DQogICAgICAgICAgICBuID0gc2VsZi5wYXJlbnROb2RlLl9ub2RlLmNoaWxkcmVuW2kgLSAxXQ0K ICAgICAgICAgICAgcmV0dXJuIE5PREVfQ0xBU1NbIG4udHlwZSBdIChuLCBzZWxmLCBzZWxmLl9k b2N1bWVudCkNCg0KICAgIGRlZiBnZXRfbmV4dFNpYmxpbmcoc2VsZik6DQogICAgICAgIGlmIHNl bGYucGFyZW50Tm9kZSBpcyBOb25lOiByZXR1cm4gTm9uZQ0KICAgICAgICBMID0gc2VsZi5wYXJl bnROb2RlLl9ub2RlLmNoaWxkcmVuDQogICAgICAgIGkgPSBzZWxmLl9pbmRleCgpDQogICAgICAg IGlmIGkgPT0gLTEgb3IgaSA9PSBsZW4oTCkgLSAxOg0KICAgICAgICAgICAgcmV0dXJuIE5vbmUN CiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIHJldHVybiBMW2kgKyAxXQ0KDQogICAgZGVmIGdl dF9hdHRyaWJ1dGVzKHNlbGYpOg0KICAgICAgICByZXR1cm4gc2VsZi5fbm9kZS5hdHRyaWJ1dGVz DQoNCiAgICBkZWYgZ2V0X293bmVyRG9jdW1lbnQoc2VsZik6DQogICAgICAgIHJldHVybiBzZWxm Ll9kb2N1bWVudA0KDQogICAgIyBNZXRob2RzDQoNCiAgICBkZWYgaW5zZXJ0QmVmb3JlKHNlbGYs IG5ld0NoaWxkLCByZWZDaGlsZCk6DQogICAgICAgIGlmIHNlbGYucmVhZG9ubHk6DQogICAgICAg ICAgICByYWlzZSBOb01vZGlmaWNhdGlvbkFsbG93ZWRFeGNlcHRpb24sICJSZWFkLW9ubHkgbm9k ZSAiK3JlcHIoc2VsZikNCiAgICAgICAgc2VsZi5fY2hlY2tDaGlsZChuZXdDaGlsZCwgc2VsZikN Cg0KICAgICAgICAjIElmIG5ld0NoaWxkIGlzIGFscmVhZHkgaW4gdGhlIHRyZWUsIHJlbW92ZSBp dA0KICAgICAgICBpZiBuZXdDaGlsZC5wYXJlbnROb2RlICE9IE5vbmU6DQogICAgICAgICAgICBu ZXdDaGlsZC5wYXJlbnROb2RlLnJlbW92ZUNoaWxkKCBuZXdDaGlsZCApDQoNCiAgICAgICAgaWYg cmVmQ2hpbGQgPT0gTm9uZToNCiAgICAgICAgICAgIHNlbGYuX25vZGUuY2hpbGRyZW4uYXBwZW5k KCBuZXdDaGlsZC5fbm9kZSApDQogICAgICAgICAgICBuZXdDaGlsZC5wYXJlbnROb2RlID0gc2Vs Zg0KICAgICAgICAgICAgcmV0dXJuIG5ld0NoaWxkDQoNCiAgICAgICAgTCA9IHNlbGYuX25vZGUu Y2hpbGRyZW4gOyBuID0gcmVmQ2hpbGQuX25vZGUNCiAgICAgICAgZm9yIGkgaW4gcmFuZ2UobGVu KEwpKToNCiAgICAgICAgICAgIGlmIExbaV0gPT0gbjoNCiAgICAgICAgICAgICAgICBMW2k6aV0g PSBbbmV3Q2hpbGQuX25vZGVdDQogICAgICAgICAgICAgICAgbmV3Q2hpbGQucGFyZW50Tm9kZSA9 IHNlbGYNCiAgICAgICAgICAgICAgICByZXR1cm4gbmV3Q2hpbGQNCiAgICAgICAgcmFpc2UgTm90 Rm91bmRFeGNlcHRpb24oInJlZkNoaWxkIG5vdCBhIGNoaWxkIGluIGluc2VydEJlZm9yZSgpIikN Cg0KICAgIGRlZiByZXBsYWNlQ2hpbGQoc2VsZiwgbmV3Q2hpbGQsIG9sZENoaWxkKToNCiAgICAg ICAgaWYgc2VsZi5yZWFkb25seToNCiAgICAgICAgICAgIHJhaXNlIE5vTW9kaWZpY2F0aW9uQWxs b3dlZEV4Y2VwdGlvbiwgIlJlYWQtb25seSBub2RlICIrcmVwcihzZWxmKQ0KICAgICAgICBzZWxm Ll9jaGVja0NoaWxkKG5ld0NoaWxkLCBzZWxmKQ0KDQogICAgICAgIG8gPSBvbGRDaGlsZC5fbm9k ZSA7IEwgPSBzZWxmLl9ub2RlLmNoaWxkcmVuDQogICAgICAgIGZvciBpIGluIHJhbmdlKGxlbihM KSk6DQogICAgICAgICAgICBpZiBMW2ldID09IG86DQogICAgICAgICAgICAgICAgIyBJZiBuZXdD aGlsZCBpcyBhbHJlYWR5IGluIHRoZSB0cmVlLCByZW1vdmUgaXQNCiAgICAgICAgICAgICAgICBp ZiBuZXdDaGlsZC5wYXJlbnROb2RlICE9IE5vbmU6DQogICAgICAgICAgICAgICAgICAgIG5ld0No aWxkLnBhcmVudE5vZGUucmVtb3ZlQ2hpbGQoIG5ld0NoaWxkICkNCg0KICAgICAgICAgICAgICAg IExbaV0gPSBuZXdDaGlsZC5fbm9kZQ0KICAgICAgICAgICAgICAgIG5ld0NoaWxkLnBhcmVudE5v ZGUgPSBzZWxmDQogICAgICAgICAgICAgICAgb2xkQ2hpbGQucGFyZW50Tm9kZSA9IE5vbmUNCiAg ICAgICAgICAgICAgICByZXR1cm4gb2xkQ2hpbGQNCiAgICAgICAgcmFpc2UgTm90Rm91bmRFeGNl cHRpb24oIm9sZENoaWxkIG5vdCBhIGNoaWxkIG9mIHRoaXMgbm9kZSIpDQoNCiAgICBkZWYgcmVt b3ZlQ2hpbGQoc2VsZiwgb2xkQ2hpbGQpOg0KICAgICAgICBpZiBzZWxmLnJlYWRvbmx5Og0KICAg ICAgICAgICAgcmFpc2UgTm9Nb2RpZmljYXRpb25BbGxvd2VkRXhjZXB0aW9uLCAiUmVhZC1vbmx5 IG5vZGUgIityZXByKHNlbGYpDQoNCiAgICAgICAgdHJ5Og0KICAgICAgICAgICAgc2VsZi5fbm9k ZS5jaGlsZHJlbi5yZW1vdmUob2xkQ2hpbGQuX25vZGUpDQogICAgICAgICAgICBvbGRDaGlsZC5w YXJlbnROb2RlID0gTm9uZQ0KICAgICAgICAgICAgcmV0dXJuIG9sZENoaWxkDQogICAgICAgIGV4 Y2VwdCBWYWx1ZUVycm9yOg0KICAgICAgICAgICAgcmFpc2UgTm90Rm91bmRFeGNlcHRpb24oIm9s ZENoaWxkIGlzIG5vdCBhIGNoaWxkIG9mIHRoaXMgbm9kZSIpDQoNCiAgICBkZWYgYXBwZW5kQ2hp bGQoc2VsZiwgbmV3Q2hpbGQpOg0KICAgIAlzZWxmLmluc2VydEJlZm9yZShuZXdDaGlsZCxOb25l KQ0KDQogICAgZGVmIGhhc0NoaWxkTm9kZXMoc2VsZik6DQogICAgICAgIHJldHVybiBsZW4oc2Vs Zi5fbm9kZS5jaGlsZHJlbikgPiAwDQoNCiAgICBkZWYgY2xvbmVOb2RlKHNlbGYsIGRlZXApOg0K ICAgICAgICBwYXNzICMgWFhYIEkgZG9uJ3QgdW5kZXJzdGFuZCB0aGUgZXhhY3QgZGVmaW5pdGlv biBvZiBjbG9uZU5vZGUoKQ0KDQoJCQkNCmNsYXNzIENoYXJhY3RlckRhdGEoTm9kZSk6DQogICAg IyBBdHRyaWJ1dGVzDQogICAgZGVmIGdldF9kYXRhKHNlbGYpOg0KICAgICAgICByZXR1cm4gc2Vs Zi5fbm9kZS52YWx1ZQ0KICAgIA0KICAgIGRlZiBzZXRfZGF0YShzZWxmLCBkYXRhKToNCiAgICAg ICAgc2VsZi5fbm9kZS52YWx1ZSA9IGRhdGENCiAgICAgICAgDQogICAgZGVmIF9fbGVuX18oc2Vs Zik6DQogICAgICAgIHJldHVybiBsZW4oc2VsZi5fbm9kZS52YWx1ZSkNCiAgICBnZXRfbGVuZ3Ro ID0gX19sZW5fXw0KDQogICAgIyBNZXRob2RzDQogICAgZGVmIHN1YnN0cmluZ0RhdGEoc2VsZiwg b2Zmc2V0LCBjb3VudCk6DQogICAgICAgICIiIkV4dHJhY3RzIGEgcmFuZ2Ugb2YgZGF0YSBmcm9t IHRoZSBvYmplY3QuDQogICAgICAgIG9mZnNldCAtLSBzdGFydCBvZiBzdWJzdHJpbmcgdG8gZXh0 cmFjdCBjb3VudCAtLSBudW1iZXIgb2YgY2hhcmFjdGVycyB0byBleHRyYWN0IiIiDQogICAgICAg IGlmIG9mZnNldDwwOg0KICAgICAgICAgICAgcmFpc2UgSW5kZXhTaXplRXhjZXB0aW9uKCJOZWdh dGl2ZSBvZmZzZXQiKQ0KICAgICAgICBpZiBvZmZzZXQ+bGVuKHNlbGYuX25vZGUudmFsdWUpOg0K ICAgICAgICAgICAgcmFpc2UgSW5kZXhTaXplRXhjZXB0aW9uKCJPZmZzZXQgbGFyZ2VyIHRoYW4g c2l6ZSBvZiBkYXRhIikNCiAgICAgICAgaWYgY291bnQ8MDoNCiAgICAgICAgICAgIHJhaXNlIElu ZGV4U2l6ZUV4Y2VwdGlvbigiTmVnYXRpdmUtbGVuZ3RoIHN1YnN0cmluZyByZXF1ZXN0ZWQiKQ0K DQogICAgICAgIHJldHVybiBzZWxmLl9ub2RlLnZhbHVlW29mZnNldDpvZmZzZXQrY291bnRdDQoN CiAgICBkZWYgYXBwZW5kRGF0YShzZWxmLCBhcmcpOg0KICAgICAgICAiIiJBcHBlbmQgdGhlIHN0 cmluZyB0byB0aGUgZW5kIG9mIHRoZSBjaGFyYWN0ZXIgZGF0YS4iIiINCiAgICAgICAgaWYgc2Vs Zi5yZWFkb25seToNCiAgICAgICAgICAgIHJhaXNlIE5vTW9kaWZpY2F0aW9uQWxsb3dlZEV4Y2Vw dGlvbigiUmVhZC1vbmx5IG9iamVjdCIpDQogICAgICAgIHNlbGYuX25vZGUudmFsdWUgPSBzZWxm Ll9ub2RlLnZhbHVlICsgYXJnDQoNCiAgICBkZWYgaW5zZXJ0RGF0YShzZWxmLCBvZmZzZXQsIGFy Zyk6DQogICAgICAgICIiIkluc2VydCBhIHN0cmluZyBhdCB0aGUgc3BlY2lmaWVkIGNoYXJhY3Rl ciBvZmZzZXQuDQogICAgICAgIG9mZnNldCAtLSBjaGFyYWN0ZXIgb2Zmc2V0IGF0IHdoaWNoIHRv IGluc2VydA0KICAgICAgICBhcmcgLS0gdGhlIHN0cmluZyB0byBpbnNlcnQiIiINCiAgICAgICAg aWYgb2Zmc2V0PDA6DQogICAgICAgICAgICByYWlzZSBJbmRleFNpemVFeGNlcHRpb24oIk5lZ2F0 aXZlIG9mZnNldCIpDQogICAgICAgIGlmIHNlbGYucmVhZG9ubHk6DQogICAgICAgICAgICByYWlz ZSBOb01vZGlmaWNhdGlvbkFsbG93ZWRFeGNlcHRpb24oIlJlYWQtb25seSBvYmplY3QiKQ0KICAg ICAgICBpZiBvZmZzZXQ+bGVuKHNlbGYuX25vZGUudmFsdWUpOg0KICAgICAgICAgICAgcmFpc2Ug SW5kZXhTaXplRXhjZXB0aW9uKCJPZmZzZXQgbGFyZ2VyIHRoYW4gc2l6ZSBvZiBkYXRhIikNCiAg ICAgICAgc2VsZi5fbm9kZS52YWx1ZSA9IHNlbGYuX25vZGUudmFsdWVbOm9mZnNldF0gKyBhcmcg KyBzZWxmLl9ub2RlLnZhbHVlW29mZnNldDpdDQoNCiAgICBkZWYgZGVsZXRlRGF0YShzZWxmLCBv ZmZzZXQsIGNvdW50KToNCiAgICAgICAgIiIiUmVtb3ZlIGEgcmFuZ2Ugb2YgY2hhcmFjdGVycyBm cm9tIHRoZSBub2RlLg0KICAgICAgICBvZmZzZXQgLS0gc3RhcnQgb2Ygc3Vic3RyaW5nIHRvIGRl bGV0ZQ0KICAgICAgICBjb3VudCAtLSBudW1iZXIgb2YgY2hhcmFjdGVycyB0byBkZWxldGUiIiIN CiAgICAgICAgaWYgb2Zmc2V0PDA6DQogICAgICAgICAgICByYWlzZSBJbmRleFNpemVFeGNlcHRp b24oIk5lZ2F0aXZlIG9mZnNldCIpDQogICAgICAgIGlmIG9mZnNldD5sZW4oc2VsZi5fbm9kZS52 YWx1ZSk6DQogICAgICAgICAgICByYWlzZSBJbmRleFNpemVFeGNlcHRpb24oIk9mZnNldCBsYXJn ZXIgdGhhbiBzaXplIG9mIGRhdGEiKQ0KICAgICAgICBpZiBzZWxmLnJlYWRvbmx5Og0KICAgICAg ICAgICAgcmFpc2UgTm9Nb2RpZmljYXRpb25BbGxvd2VkRXhjZXB0aW9uKCJSZWFkLW9ubHkgb2Jq ZWN0IikNCiAgICAgICAgc2VsZi5fbm9kZS52YWx1ZSA9IHNlbGYuX25vZGUudmFsdWVbOm9mZnNl dF0gKyBzZWxmLl9ub2RlLnZhbHVlW29mZnNldCtjb3VudDpdICAgICAgICANCg0KICAgIGRlZiBy ZXBsYWNlRGF0YShzZWxmLCBvZmZzZXQsIGNvdW50LCBhcmcpOg0KICAgICAgICAiIiJSZXBsYWNl IHRoZSBjaGFyYWN0ZXJzIHN0YXJ0aW5nIGF0IHRoZSBzcGVjaWZpZWQgb2Zmc2V0DQogICAgICAg IHdpdGggdGhlIHNwZWNpZmllZCBzdHJpbmcuDQogICAgICAgIG9mZnNldCAtLSBzdGFydCBvZiBz dWJzdHJpbmcgdG8gZGVsZXRlDQogICAgICAgIGNvdW50IC0tIG51bWJlciBvZiBjaGFyYWN0ZXJz IHRvIGRlbGV0ZQ0KICAgICAgICBhcmcgLS0gc3RyaW5nIHdpdGggd2hpY2ggdGhlIHJhbmdlIHdp bGwgYmUgcmVwbGFjZWQiIiINCiAgICAgICAgaWYgb2Zmc2V0PDA6DQogICAgICAgICAgICByYWlz ZSBJbmRleFNpemVFeGNlcHRpb24oIk5lZ2F0aXZlIG9mZnNldCIpDQogICAgICAgIGlmIG9mZnNl dD5sZW4oc2VsZi5fbm9kZS52YWx1ZSk6DQogICAgICAgICAgICByYWlzZSBJbmRleFNpemVFeGNl cHRpb24oIk9mZnNldCBsYXJnZXIgdGhhbiBzaXplIG9mIGRhdGEiKQ0KICAgICAgICBpZiBzZWxm LnJlYWRvbmx5Og0KICAgICAgICAgICAgcmFpc2UgTm9Nb2RpZmljYXRpb25BbGxvd2VkRXhjZXB0 aW9uKCJSZWFkLW9ubHkgb2JqZWN0IikNCiAgICAgICAgc2VsZi5fbm9kZS52YWx1ZSA9IHNlbGYu X25vZGUudmFsdWVbOm9mZnNldF0gKyBhcmcgKyBzZWxmLl9ub2RlLnZhbHVlW29mZnNldCtjb3Vu dDpdDQoNCiAgICAjIFhYWCBkZWZpbmUgX19nZXRzbGljZV9fICYgZnJpZW5kcyBoZXJlLi4uDQoN CiAgICBkZWYgdG94bWwoc2VsZik6DQogICAgICAgIHJldHVybiBlc2NhcGUoc2VsZi5fbm9kZS52 YWx1ZSkgDQogICAgDQpjbGFzcyBBdHRyKE5vZGUpOg0KICAgIGRlZiBfX2luaXRfXyhzZWxmLCBu b2RlLCBwYXJlbnQgPSBOb25lKToNCiAgICAgICAgTm9kZS5fX2luaXRfXyhzZWxmLCBub2RlLCBw YXJlbnQsIE5vbmUpDQoNCiAgICBkZWYgX19yZXByX18oc2VsZik6DQogICAgICAgIHJldHVybiAn PEF0dHJpYnV0ZSBub2RlICVzLCAlcz4nICUgKHJlcHIoc2VsZi5fbm9kZS5uYW1lKSwNCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcmVwcihzZWxmLl9ub2RlLnZh bHVlKSkNCg0KICAgIGRlZiBnZXRfbmFtZShzZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYuX25v ZGUubmFtZQ0KDQogICAgZGVmIGdldF92YWx1ZShzZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYu X25vZGUudmFsdWUNCg0KICAgIGRlZiBzZXRfdmFsdWUoc2VsZiwgdmFsdWUpOg0KICAgICAgICBz ZWxmLl9ub2RlLnZhbHVlID0gdmFsdWUNCiAgICAgICAgc2VsZi5fbm9kZS5zcGVjaWZpZWQgPSAx DQoNCiAgICBkZWYgZ2V0X3NwZWNpZmllZChzZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYuX25v ZGUuc3BlY2lmaWVkDQoNCmNsYXNzIEVsZW1lbnQoTm9kZSk6DQogICAgZGVmIF9faW5pdF9fKHNl bGYsIG5vZGUsIHBhcmVudCA9IE5vbmUsIGRvY3VtZW50ID0gTm9uZSk6DQogICAgICAgIE5vZGUu X19pbml0X18oc2VsZiwgbm9kZSwgcGFyZW50LCBkb2N1bWVudCkNCiAgICAgICAgDQogICAgZGVm IF9fcmVwcl9fKHNlbGYpOg0KICAgICAgICByZXR1cm4gIjxFbGVtZW50ICclcyc+IiAlIChzZWxm Ll9ub2RlLm5hbWUpDQoNCiAgICBkZWYgdG94bWwoc2VsZik6DQogICAgICAgIHMgPSAiPCIgKyBz ZWxmLl9ub2RlLm5hbWUNCiAgICAgICAgZm9yIG5hbWUsIHZhbHVlIGluIHNlbGYuX25vZGUuYXR0 cmlidXRlcy5pdGVtcygpOg0KICAgICAgICAgICAgcyA9IHMgKyAiICVzPSclcyciICUgKG5hbWUs IHZhbHVlKQ0KICAgICAgICBpZiBsZW4oc2VsZi5fbm9kZS5jaGlsZHJlbikgPT0gMDoNCiAgICAg ICAgICAgIHJldHVybiBzICsgIi8+Ig0KICAgICAgICBzID0gcyArICc+Jw0KICAgICAgICBmb3Ig Y2hpbGQgaW4gc2VsZi5fbm9kZS5jaGlsZHJlbjoNCiAgICAgICAgICAgIG4gPSBOT0RFX0NMQVNT WyBjaGlsZC50eXBlIF0gKGNoaWxkLCBzZWxmLCBzZWxmLl9kb2N1bWVudCkNCiAgICAgICAgICAg IHMgPSBzICsgbi50b3htbCgpDQogICAgICAgIHMgPSBzICsgIjwvIiArIHNlbGYuX25vZGUubmFt ZSArICc+Jw0KICAgICAgICByZXR1cm4gcw0KDQogICAgIyBBdHRyaWJ1dGVzDQogICAgDQogICAg ZGVmIGdldF90YWdOYW1lKHNlbGYpOg0KICAgICAgICByZXR1cm4gc2VsZi5fbm9kZS5uYW1lDQoN CiAgICAjIE1ldGhvZHMNCiAgICANCiAgICBkZWYgZ2V0QXR0cmlidXRlKHNlbGYsIG5hbWUpOg0K ICAgICAgICBpZiBzZWxmLl9ub2RlLmF0dHJpYnV0ZXMuaGFzX2tleShuYW1lKToNCiAgICAgICAg ICAgIHJldHVybiBzZWxmLl9ub2RlLmF0dHJpYnV0ZXNbbmFtZV0NCiAgICAgICAgZWxzZToNCiAg ICAgICAgICAgIHJldHVybiAiIg0KICAgIA0KICAgIGRlZiBzZXRBdHRyaWJ1dGUoc2VsZiwgbmFt ZSwgdmFsdWUpOg0KICAgICAgICBzZWxmLl9ub2RlLmF0dHJpYnV0ZXNbbmFtZV0gPSB2YWx1ZQ0K DQogICAgZGVmIHJlbW92ZUF0dHJpYnV0ZShzZWxmLCBuYW1lKToNCiAgICAgICAgaWYgc2VsZi5f bm9kZS5hdHRyaWJ1dGVzLmhhc19rZXkobmFtZSk6DQogICAgICAgICAgICBkZWwgc2VsZi5fbm9k ZS5hdHRyaWJ1dGVzW25hbWVdDQoNCiAgICBkZWYgZ2V0QXR0cmlidXRlTm9kZShzZWxmLCBuYW1l KToNCiAgICAgICAgaWYgbm90IHNlbGYuX25vZGUuYXR0cmlidXRlcy5oYXNfa2V5WyBuYW1lIF06 DQogICAgICAgICAgICByZXR1cm4gTm9uZQ0KICAgICAgICBkID0gX25vZGVEYXRhKEFUVFJJQlVU RV9OT0RFKQ0KICAgICAgICBkLm5hbWUgPSBuYW1lDQogICAgICAgIGQudmFsdWUgPSBzZWxmLl9u b2RlLmF0dHJpYnV0ZXNbbmFtZV0NCiAgICAgICAgZC5hdHRyaWJ1dGVzID0gTm9uZQ0KICAgICAg ICByZXR1cm4gQXR0cihkLCBOb25lKQ0KDQogICAgZGVmIHNldEF0dHJpYnV0ZU5vZGUoc2VsZiwg bmV3QXR0cik6DQogICAgICAgIGlmIHNlbGYuX25vZGUuYXR0cmlidXRlcy5oYXNfa2V5WyBuZXdB dHRyLl9ub2RlLm5hbWUgXToNCiAgICAgICAgICAgIGQgPSBfbm9kZURhdGEoQVRUUklCVVRFX05P REUpDQogICAgICAgICAgICBkLm5hbWUgPSBuZXdBdHRyLl9ub2RlLm5hbWUNCiAgICAgICAgICAg IGQudmFsdWUgPSBzZWxmLl9ub2RlLmF0dHJpYnV0ZXNbIG5ld0F0dHIuX25vZGUubmFtZV0NCiAg ICAgICAgICAgIGQuYXR0cmlidXRlcyA9IE5vbmUNCiAgICAgICAgICAgIHJldHZhbCA9IEF0dHIo ZCwgTm9uZSkNCiAgICAgICAgZWxzZTogcmV0dmFsID0gTm9uZQ0KDQogICAgICAgIHNlbGYuX25v ZGUuYXR0cmlidXRlc1sgbmV3QXR0ci5fbm9kZS5uYW1lIF0gPSBuZXdBdHRyLl9ub2RlLnZhbHVl DQogICAgICAgIHJldHVybiByZXR2YWwNCg0KICAgIGRlZiByZW1vdmVBdHRyaWJ1dGVOb2RlKHNl bGYsIG9sZEF0dHIpOg0KICAgICAgICAjIFhYWCB0aGlzIG5lZWRzIHRvIGtub3cgYWJvdXQgRFRE cyB0byByZXN0b3JlIGEgZGVmYXVsdCB2YWx1ZQ0KICAgICAgICBpZiBzZWxmLl9ub2RlLmF0dHJp YnV0ZXMuaGFzX2tleVsgb2xkQXR0ci5fbm9kZS5uYW1lIF06DQogICAgICAgICAgICBkID0gX25v ZGVEYXRhKEFUVFJJQlVURV9OT0RFKQ0KICAgICAgICAgICAgZC5uYW1lID0gbmV3QXR0ci5fbm9k ZS5uYW1lDQogICAgICAgICAgICBkLnZhbHVlID0gc2VsZi5fbm9kZS5hdHRyaWJ1dGVzWyBvbGRB dHRyLl9ub2RlLm5hbWUgXQ0KICAgICAgICAgICAgZC5hdHRyaWJ1dGVzID0gTm9uZQ0KICAgICAg ICAgICAgcmV0dmFsID0gQXR0cihkLCBOb25lKQ0KICAgICAgICAgICAgZGVsIHNlbGYuX25vZGUu YXR0cmlidXRlc1sgZC5uYW1lIF0NCiAgICAgICAgICAgIHJldHVybiByZXR2YWwNCiAgICAgICAg ZWxzZTogcmV0dXJuIE5vbmUNCg0KICAgIGRlZiBnZXRFbGVtZW50c0J5VGFnTmFtZShzZWxmLCB0 YWduYW1lKToNCiAgICAgICAgTCA9IFtdDQogICAgICAgIGZvciBjaGlsZCBpbiBzZWxmLl9ub2Rl LmNoaWxkcmVuOg0KICAgICAgICAgICAgaWYgY2hpbGQudHlwZSA9PSBFTEVNRU5UOg0KICAgICAg ICAgICAgICAgIGQgPSBFbGVtZW50KGNoaWxkLCBzZWxmLCBzZWxmLl9kb2N1bWVudCkNCiAgICAg ICAgICAgICAgICBpZiBjaGlsZC5uYW1lID09IHRhZ25hbWU6DQogICAgICAgICAgICAgICAgICAg IEwuYXBwZW5kKGQpDQogICAgICAgICAgICAgICAgTCA9IEwgKyBkLmdldEVsZW1lbnRzQnlUYWdO YW1lKHRhZ25hbWUpDQogICAgICAgIHJldHVybiBOb2RlTGlzdChMKQ0KDQogICAgZGVmIG5vcm1h bGl6ZShzZWxmKToNCiAgICAgICAgIyBUcmF2ZXJzZSB0aGUgbGlzdCBvZiBjaGlsZHJlbiwgYW5k IG1lcmdlIGFkamFjZW50IHRleHQgbm9kZXMNCiAgICAgICAgTCA9IHNlbGYuX25vZGUuY2hpbGRy ZW4NCiAgICAgICAgZm9yIGkgaW4gcmFuZ2UobGVuKEwpLTEsIDAsIC0xKToNCiAgICAgICAgICAg IGlmIExbaV0udHlwZSA9PSBURVhUX05PREUgYW5kIExbaS0xXS50eXBlID09IFRFWFRfTk9ERToN CiAgICAgICAgICAgICAgICAjIFR3byB0ZXh0IG5vZGVzIHRvZ2V0aGVyLCBzbyBtZXJnZSB0aGVt DQogICAgICAgICAgICAgICAgIyBYWFggYW55IFRleHQgaW5zdGFuY2VzIHByb3h5aW5nIHRoZSBk ZWxldGVkDQogICAgICAgICAgICAgICAgIyBfbm9kZURhdGEgaW5zdGFuY2Ugd2lsbCBmaW5kIHRo ZW1zZWx2ZXMNCiAgICAgICAgICAgICAgICAjIGRpc2Nvbm5lY3RlZCBmcm9tIHRoZSB0cmVlLiAg DQogICAgICAgICAgICAgICAgTFtpLTFdLnZhbHVlID0gTFtpLTFdLnZhbHVlICsgTFtpXS52YWx1 ZQ0KICAgICAgICAgICAgICAgIGRlbCBMW2k6aSsxXQ0KICAgICAgICAgICAgICAgIA0KICAgICAg ICBmb3IgaSBpbiByYW5nZShsZW4oTCkpOg0KICAgICAgICAgICAgaWYgTFtpXS50eXBlID09IEVM RU1FTlRfTk9ERToNCiAgICAgICAgICAgICAgICBuID0gTk9ERV9DTEFTU1sgTFtpXS50eXBlIF0g KExbaV0sIHNlbGYsIHNlbGYuX2RvY3VtZW50KQ0KICAgICAgICAgICAgICAgIG4ubm9ybWFsaXpl KCkNCiAgICANCmNsYXNzIFRleHQoQ2hhcmFjdGVyRGF0YSk6DQogICAgIyBNZXRob2RzDQoNCiAg ICBkZWYgX19yZXByX18oc2VsZik6DQogICAgICAgIGlmIGxlbihzZWxmLl9ub2RlLnZhbHVlKTwy MDogcz1zZWxmLl9ub2RlLnZhbHVlDQogICAgICAgIGVsc2U6IHM9c2VsZi5fbm9kZS52YWx1ZVs6 MTddICsgJy4uLicNCiAgICAgICAgcmV0dXJuICc8VGV4dCBub2RlICVzPicgJSAocmVwcihzKSwp DQogICAgICAgIA0KICAgIGRlZiBzcGxpdFRleHQoc2VsZiwgb2Zmc2V0KToNCiAgICAgICAgbjEg PSBfbm9kZURhdGEoVEVYVF9OT0RFKSA7IG4yID0gX25vZGVEYXRhKFRFWFRfTk9ERSkNCiAgICAg ICAgbjEubmFtZSA9ICIjdGV4dCIgOyBuMi5uYW1lID0gIiN0ZXh0Ig0KICAgICAgICBuMS52YWx1 ZSA9IHNlbGYuc3Vic3RyaW5nRGF0YSgwLCBvZmZzZXQpDQogICAgICAgIG4yLnZhbHVlID0gc2Vs Zi5zdWJzdHJpbmdEYXRhKG9mZnNldCwgbGVuKHNlbGYpIC0gb2Zmc2V0KQ0KICAgICAgICBwYXJl bnQgPSBzZWxmLnBhcmVudE5vZGUNCiAgICAgICAgbjEgPSBUZXh0KG4xLCBOb25lLCBzZWxmLl9k b2N1bWVudCkNCiAgICAgICAgbjIgPSBUZXh0KG4yLCBOb25lLCBzZWxmLl9kb2N1bWVudCkNCg0K ICAgICAgICAjIEluc2VydCBuMSBhbmQgbjIsIGFuZCBkZWxldGUgdGhpcyBub2RlDQogICAgICAg IHBhcmVudC5pbnNlcnRCZWZvcmUobjEsIHNlbGYpDQogICAgICAgIHBhcmVudC5yZXBsYWNlQ2hp bGQobjIsIHNlbGYpDQogICAgDQpjbGFzcyBDb21tZW50KENoYXJhY3RlckRhdGEpOg0KICAgIGRl ZiBfX3JlcHJfXyhzZWxmKToNCiAgICAgICAgaWYgbGVuKHNlbGYuX25vZGUudmFsdWUpPDIwOiBz PXNlbGYuX25vZGUudmFsdWUNCiAgICAgICAgZWxzZTogcz1zZWxmLl9ub2RlLnZhbHVlWzoxN10g KyAnLi4uJw0KICAgICAgICByZXR1cm4gJzxDb21tZW50IG5vZGUgJXM+JyAlIChyZXByKHMpLCkN CiAgICANCiAgICBkZWYgdG94bWwoc2VsZik6DQogICAgICAgIHJldHVybiAnPC0tICVzIC0tPicg JSBzZWxmLl9ub2RlLnZhbHVlDQoNCmNsYXNzIENEQVRBU2VjdGlvbihUZXh0KToNCiAgICAiIiJS ZXByZXNlbnRzIENEQVRBIHNlY3Rpb25zLCB3aGljaCBhcmUgYmxvY2tzIG9mIHRleHQgdGhhdCB3 b3VsZA0KICAgIG90aGVyd2lzZSBiZSByZWdhcmRlZCBhcyBtYXJrdXAuIiIiDQogICAgZGVmIF9f cmVwcl9fKHNlbGYpOg0KICAgICAgICBpZiBsZW4oc2VsZi5fbm9kZS52YWx1ZSk8MjA6IHM9c2Vs Zi5fbm9kZS52YWx1ZQ0KICAgICAgICBlbHNlOiBzPXNlbGYuX25vZGUudmFsdWVbOjE3XSArICcu Li4nDQogICAgICAgIHJldHVybiAnPENEQVRBU2VjdGlvbiBub2RlICVzPicgJSAocmVwcihzKSwp DQoNCiAgICBkZWYgdG94bWwoc2VsZik6DQogICAgICAgIHJldHVybiBzZWxmLl9ub2RlLnZhbHVl DQoNCmNsYXNzIERvY3VtZW50VHlwZShOb2RlKToNCiAgICByZWFkb25seSA9IDEgICAgIyBUaGlz IGlzIGEgcmVhZC1vbmx5IGNsYXNzDQoNCiAgICAjIEF0dHJpYnV0ZXMNCiAgICBkZWYgZ2V0X25h bWUoc2VsZik6DQogICAgICAgIHJldHVybiBzZWxmLl9ub2RlLm5hbWUNCg0KICAgIGRlZiBnZXRf ZW50aXRpZXMoc2VsZik6DQogICAgICAgIHBhc3MgIyBYWFgNCg0KICAgIGRlZiBnZXRfbm90YXRp b25zKHNlbGYpOg0KICAgICAgICBwYXNzICMgWFhYDQoNCmNsYXNzIE5vdGF0aW9uKE5vZGUpOg0K ICAgIHJlYWRvbmx5ID0gMSAgICAjIFRoaXMgaXMgYSByZWFkLW9ubHkgY2xhc3MNCiAgICAjIEF0 dHJpYnV0ZXMNCiAgICBkZWYgZ2V0X3B1YmxpY0lkKHNlbGYpOg0KICAgICAgICBwYXNzICMgWFhY DQogICAgICAgIA0KICAgIGRlZiBnZXRfc3lzdGVtSWQoc2VsZik6DQogICAgICAgIHBhc3MgIyBY WFgNCiAgICAgICAgDQpjbGFzcyBFbnRpdHkoTm9kZSk6DQogICAgcmVhZG9ubHkgPSAxICAgICMg VGhpcyBpcyBhIHJlYWQtb25seSBjbGFzcw0KICAgIGRlZiBnZXRfcHVibGljSWQoc2VsZik6DQog ICAgICAgIHBhc3MgIyBYWFgNCg0KICAgIGRlZiBnZXRfc3lzdGVtSWQoc2VsZik6DQogICAgICAg IHBhc3MgIyBYWFgNCg0KICAgIGRlZiBnZXRfbm90YXRpb25OYW1lKHNlbGYpOg0KICAgICAgICBw YXNzICMgWFhYDQoNCmNsYXNzIEVudGl0eVJlZmVyZW5jZShOb2RlKToNCiAgICBwYXNzIA0KDQpj bGFzcyBQcm9jZXNzaW5nSW5zdHJ1Y3Rpb24oTm9kZSk6DQogICAgZGVmIHRveG1sKHNlbGYpOg0K ICAgICAgICByZXR1cm4gIjw/ICIgKyBzZWxmLl9ub2RlLm5hbWUgKyAnICcgK3NlbGYuX25vZGUu dGFyZ2V0ICsgIj8+Ig0KDQogICAgZGVmIGdldF90YXJnZXQoc2VsZik6DQogICAgICAgIHJldHVy biBzZWxmLl9ub2RlLm5hbWUNCg0KICAgIGRlZiBnZXRfZGF0YShzZWxmKToNCiAgICAgICAgcmV0 dXJuIHNlbGYuX25vZGUudGFyZ2V0DQoNCiAgICBkZWYgc2V0X2RhdGEoc2VsZiwgZGF0YSk6DQog ICAgICAgIHNlbGYuX25vZGUudGFyZ2V0ID0gZGF0YQ0KDQoNCmNsYXNzIERvY3VtZW50KE5vZGUp Og0KICAgIGRlZiBfX2luaXRfXyhzZWxmLCBub2RlLCBwYXJlbnQgPSBOb25lLCBkb2N1bWVudCA9 IE5vbmUpOg0KICAgICAgICBOb2RlLl9faW5pdF9fKHNlbGYsIG5vZGUsIHBhcmVudCA9IE5vbmUs IGRvY3VtZW50ID0gTm9uZSkNCiAgICAgICAgc2VsZi5kb2N1bWVudFR5cGUgPSBOb25lDQogICAg ICAgICNzZWxmLmRvY3VtZW50RWxlbWVudCA9IE5vbmUNCiAgICAgICAgc2VsZi5ET01JbXBsZW1l bnRhdGlvbiA9IF9faW1wb3J0X18oX19uYW1lX18pDQoNCiAgICBkZWYgdG94bWwoc2VsZik6DQog ICAgICAgIHMgPSAnPD94bWwgdmVyc2lvbj0iMS4wIj8+XG4nDQogICAgICAgIHMgPSBzICsgIjwh RE9DVFlQRSBYWFg+XG4iDQogICAgICAgIGlmIGxlbihzZWxmLl9ub2RlLmNoaWxkcmVuKToNCiAg ICAgICAgICAgIG4gPSBzZWxmLl9ub2RlLmNoaWxkcmVuWzBdDQogICAgICAgICAgICBuID0gIE5P REVfQ0xBU1NbIG4udHlwZSBdIChuLCBzZWxmLCBzZWxmLl9kb2N1bWVudCkNCiAgICAgICAgICAg IHMgPSBzICsgbi50b3htbCgpDQogICAgICAgIHJldHVybiBzDQoNCiAgICBkZWYgY3JlYXRlRWxl bWVudChzZWxmLCB0YWdOYW1lKToNCiAgICAgICAgZCA9IF9ub2RlRGF0YShFTEVNRU5UX05PREUp DQogICAgICAgIGQubmFtZSA9IHRhZ05hbWUNCiAgICAgICAgZC52YWx1ZSA9IE5vbmUNCiAgICAg ICAgZC5hdHRyaWJ1dGVzID0gTmFtZWROb2RlTWFwKCkNCiAgICAgICAgcmV0dXJuIEVsZW1lbnQo ZCwgTm9uZSwgc2VsZikNCg0KICAgIGRlZiBjcmVhdGVEb2N1bWVudEZyYWdtZW50KHNlbGYpOg0K ICAgICAgICBwYXNzICNYWFgNCg0KICAgIGRlZiBjcmVhdGVUZXh0Tm9kZShzZWxmLCBkYXRhKToN CiAgICAgICAgZCA9IF9ub2RlRGF0YShURVhUX05PREUpDQogICAgICAgIGQubmFtZSA9ICIjdGV4 dCINCiAgICAgICAgZC52YWx1ZSA9IGRhdGENCiAgICAgICAgcmV0dXJuIFRleHQoZCwgTm9uZSwg c2VsZikNCg0KICAgIGRlZiBjcmVhdGVDb21tZW50KHNlbGYsIGRhdGEpOg0KICAgICAgICBkID0g X25vZGVEYXRhKENPTU1FTlRfTk9ERSkNCiAgICAgICAgZC5uYW1lID0gIiNjb21tZW50Ig0KICAg ICAgICBkLnZhbHVlID0gZGF0YQ0KICAgICAgICByZXR1cm4gQ29tbWVudChkLCBOb25lLCBzZWxm KQ0KDQogICAgZGVmIGNyZWF0ZUNEQVRBc2VjdGlvbihzZWxmLCBkYXRhKToNCiAgICAgICAgZCA9 IF9ub2RlRGF0YShDREFUQV9TRUNUSU9OX05PREUpDQogICAgICAgIGQubmFtZSA9ICIjY2RhdGEt c2VjdGlvbiINCiAgICAgICAgZC52YWx1ZSA9IGRhdGENCiAgICAgICAgcmV0dXJuIENEQVRBU2Vj dGlvbihkLCBOb25lLCBzZWxmKQ0KDQogICAgZGVmIGNyZWF0ZVByb2Nlc3NpbmdJbnN0cnVjdGlv bihzZWxmLCB0YXJnZXQsIGRhdGEpOg0KICAgICAgICBkID0gX25vZGVEYXRhKFBST0NFU1NJTkdf SU5TVFJVQ1RJT05fTk9ERSkNCiAgICAgICAgZC5uYW1lID0gdGFyZ2V0DQogICAgICAgIGQudmFs dWUgPSBkYXRhDQogICAgICAgIHJldHVybiBQcm9jZXNzaW5nSW5zdHJ1Y3Rpb24oZCwgTm9uZSwg c2VsZikNCiAgICAgICAgDQogICAgZGVmIGNyZWF0ZUF0dHJpYnV0ZShzZWxmLCBuYW1lKToNCiAg ICAgICAgZCA9IF9ub2RlRGF0YShBVFRSSUJVVEVfTk9ERSkNCiAgICAgICAgZC5uYW1lID0gbmFt ZQ0KICAgICAgICBkLnZhbHVlID0gIiINCiAgICAgICAgcmV0dXJuIEF0dHJpYnV0ZShkLCBOb25l LCBzZWxmKQ0KDQogICAgZGVmIGNyZWF0ZUVudGl0eVJlZmVyZW5jZShzZWxmLCBuYW1lKToNCiAg ICAgICAgZCA9IF9ub2RlRGF0YShFTlRJVFlfUkVGRVJFTkNFX05PREUpDQogICAgICAgIGQubmFt ZSA9IG5hbWUNCiAgICAgICAgZC52YWx1ZSA9IE5vbmUNCiAgICAgICAgcmV0dXJuIEVudGl0eVJl ZmVyZW5jZShkLCBOb25lLCBzZWxmKQ0KDQogICAgZGVmIGdldEVsZW1lbnRzQnlUYWdOYW1lKHNl bGYsIHRhZ25hbWUpOg0KICAgICAgICBkb2N1bWVudEVsZW1lbnQgPSBzZWxmLmdldF9kb2N1bWVu dEVsZW1lbnQoKQ0KICAgICAgICBMID0gW10NCiAgICAgICAgaWYgZG9jdW1lbnRFbGVtZW50ID09 IE5vbmU6IHJldHVybiBMDQogICAgICAgIGlmIGRvY3VtZW50RWxlbWVudC5fbm9kZS5uYW1lID09 IHRhZ25hbWU6DQogICAgICAgICAgICBMLmFwcGVuZCggZG9jdW1lbnRFbGVtZW50ICkNCiAgICAg ICAgcmV0dXJuIEwgKyBkb2N1bWVudEVsZW1lbnQuZ2V0RWxlbWVudHNCeVRhZ05hbWUodGFnbmFt ZSkNCg0KICAgICMgQXR0cmlidXRlcw0KICAgIGRlZiBnZXRfZG9jdHlwZShzZWxmKToNCiAgICAg ICAgcmV0dXJuIHNlbGYuZG9jdW1lbnRUeXBlDQogICAgZGVmIGdldF9pbXBsZW1lbnRhdGlvbihz ZWxmKToNCiAgICAgICAgcmV0dXJuIHNlbGYuRE9NSW1wbGVtZW50YXRpb24NCiAgICBkZWYgZ2V0 X2RvY3VtZW50RWxlbWVudChzZWxmKToNCiAgICAgICAgI3JldHVybiBzZWxmLmRvY3VtZW50RWxl bWVudA0KICAgICAgICBpZiBsZW4oc2VsZi5fbm9kZS5jaGlsZHJlbikgPiAwOg0KICAgICAgICAJ cmV0dXJuIEVsZW1lbnQoc2VsZi5fbm9kZS5jaGlsZHJlblswXSxzZWxmLHNlbGYpDQogICAgICAg IHJldHVybiBOb25lDQoNCiAgICAjIE92ZXJyaWRlIHRoZSBOb2RlIG11dGF0aW9uIG1ldGhvZHMg aW4gb3JkZXIgdG8gY2hlY2sgdGhhdA0KICAgICMgdGhlcmUncyBhdCBtb3N0IGEgc2luZ2xlIEVs ZW1lbnQgY2hpbGQsIGFuZCB0byB1cGRhdGUNCiAgICAjIHNlbGYuZG9jdW1lbnRFbGVtZW50Lg0K DQoNCiAgICAjIERvbid0IHVzZSB0aGlzIG9yIGEgY2lyY3VsYXIgcmVmZXJlbmNlIG9jY3VycyEN CiAgICAjZGVmIF9zZXREb2N1bWVudEVsZW1lbnQoc2VsZik6DQogICAgIyAgICBlbGVtZW50ID0g Tm9uZQ0KICAgICMgICAgZm9yIGVsZW0gaW4gc2VsZi5fbm9kZS5jaGlsZHJlbjoNCiAgICAjICAg ICAgICBpZiBlbGVtLnR5cGUgPT0gRUxFTUVOVF9OT0RFOg0KICAgICMgICAgICAgICAgICBpZiBl bGVtZW50IGlzIE5vbmU6DQogICAgIyAgICAgICAgICAgICAgICBlbGVtZW50ID0gRWxlbWVudChl bGVtLCBzZWxmLCBzZWxmKQ0KICAgICMgICAgICAgICAgICBlbHNlOg0KICAgICMgICAgICAgICAg ICAgICAgcmFpc2UgSGllcmFyY2h5UmVxdWVzdEV4Y2VwdGlvbiwgIlRvbyBtYW55IEVsZW1lbnQg Y2hpbGRyZW4gb2YgRG9jdW1lbnQiDQogICAgIyAgICBzZWxmLmRvY3VtZW50RWxlbWVudCA9IGVs ZW1lbnQNCg0KICAgIGRlZiBpbnNlcnRCZWZvcmUoc2VsZiwgbmV3Q2hpbGQsIHJlZkNoaWxkKToN CiAgICAgICAgIyBUaGVyZSBjYW4gb25seSBiZSBvbmUgY2hpbGQgb2YgRG9jdW1lbnQgc28gcmVm Q2hpbGQgc2hvdWxkIGJlIE5vbmUNCiAgICAgICAgIyBhbmQgdGhlcmUgc2hvdWxkIG5vdCBiZSBh biBleGlzdGluZyBjaGlsZCBvciBhdCB3b3JzdCBpdCBzaG91bGQgYmUNCiAgICAgICAgIyB0aGUg c2FtZSBjaGlsZC4NCiAgICAgICAgaWYgcmVmQ2hpbGQgIT0gTm9uZSBvciBsZW4oc2VsZi5fbm9k ZS5jaGlsZHJlbikgIT0gMCBhbmQgc2VsZi5fbm9kZS5jaGlsZHJlblswXSAhPSBuZXdDaGlsZC5f bm9kZToNCiAgICAgICAgICAgIHJhaXNlIEhpZXJhcmNoeVJlcXVlc3RFeGNlcHRpb24sIFwNCiAg ICAgICAgICAgICAgICAiQ2FuJ3QgYWRkICVzOyB0b28gbWFueSBFbGVtZW50IGNoaWxkcmVuIG9m IERvY3VtZW50IiAlIChyZXByKG5ld0NoaWxkKSwpDQogICAgICAgIAkNCiAgICAgICAgcmV0dmFs ID0gTm9kZS5pbnNlcnRCZWZvcmUoc2VsZiwgbmV3Q2hpbGQsIHJlZkNoaWxkKQ0KICAgICAgICAj c2VsZi5fc2V0RG9jdW1lbnRFbGVtZW50KCkNCiAgICAgICAgcmV0dXJuIHJldHZhbA0KDQogICAg ZGVmIHJlcGxhY2VDaGlsZChzZWxmLCBuZXdDaGlsZCwgb2xkQ2hpbGQpOg0KICAgICAgICByZXR2 YWwgPSBOb2RlLnJlcGxhY2VDaGlsZChzZWxmLCBuZXdDaGlsZCwgb2xkQ2hpbGQpDQogICAgICAg ICNzZWxmLl9zZXREb2N1bWVudEVsZW1lbnQoKQ0KICAgICAgICByZXR1cm4gcmV0dmFsDQoNCiAg ICBkZWYgcmVtb3ZlQ2hpbGQoc2VsZiwgb2xkQ2hpbGQpOg0KICAgICAgICByZXR2YWwgPSBOb2Rl LnJlbW92ZUNoaWxkKHNlbGYsIG9sZENoaWxkKQ0KICAgICAgICAjc2VsZi5fc2V0RG9jdW1lbnRF bGVtZW50KCkNCiAgICAgICAgcmV0dXJuIHJldHZhbA0KDQogICAgZGVmIGFwcGVuZENoaWxkKHNl bGYsIG5ld0NoaWxkKToNCiAgICAgICAgcmV0dmFsID0gc2VsZi5pbnNlcnRCZWZvcmUobmV3Q2hp bGQsTm9uZSkNCiAgICAgICAgcmV0dXJuIHJldHZhbA0KICAgIA0KY2xhc3MgRG9jdW1lbnRGcmFn bWVudChOb2RlKToNCiAgICBwYXNzDQoNCiMgRGljdGlvbmFyeSBtYXBwaW5nIHR5cGVzIHRvIHRo ZSBjb3JyZXNwb25kaW5nIGNsYXNzIG9iamVjdA0KDQpOT0RFX0NMQVNTID0gew0KRUxFTUVOVF9O T0RFICAgICAgICAgICAgICAgIDogRWxlbWVudCwNCkFUVFJJQlVURV9OT0RFICAgICAgICAgICAg ICA6IEF0dHIsIA0KVEVYVF9OT0RFICAgICAgICAgICAgICAgICAgIDogVGV4dCwgDQpDREFUQV9T RUNUSU9OX05PREUgICAgICAgICAgOiBDREFUQVNlY3Rpb24sDQpFTlRJVFlfUkVGRVJFTkNFX05P REUgICAgICAgOiBFbnRpdHlSZWZlcmVuY2UsDQpFTlRJVFlfTk9ERSAgICAgICAgICAgICAgICAg OiBFbnRpdHksIA0KUFJPQ0VTU0lOR19JTlNUUlVDVElPTl9OT0RFIDogUHJvY2Vzc2luZ0luc3Ry dWN0aW9uLCANCkNPTU1FTlRfTk9ERSAgICAgICAgICAgICAgICA6IENvbW1lbnQsDQpET0NVTUVO VF9OT0RFICAgICAgICAgICAgICAgOiBEb2N1bWVudCwNCkRPQ1VNRU5UX1RZUEVfTk9ERSAgICAg ICAgICA6IERvY3VtZW50VHlwZSwNCkRPQ1VNRU5UX0ZSQUdNRU5UX05PREUgICAgICA6IERvY3Vt ZW50RnJhZ21lbnQsDQpOT1RBVElPTl9OT0RFICAgICAgICAgICAgICAgOiBOb3RhdGlvbg0KfQ0K DQojIFNldCB0aGUgY2hpbGRDbGFzc2VzIGF0dHJpYnV0ZXM7IHdlIG5lZWQgdG8gZG8gdGhpcyBh dCB0aGUgZW5kIG9uY2UNCiMgYWxsIHRoZSBjbGFzc2VzIGhhdmUgYmVlbiBkZWZpbmVkLg0KDQpE b2N1bWVudC5jaGlsZENsYXNzZXMgPSBbRWxlbWVudCwgUHJvY2Vzc2luZ0luc3RydWN0aW9uLCBD b21tZW50LCBEb2N1bWVudFR5cGVdDQpEb2N1bWVudEZyYWdtZW50LmNoaWxkQ2xhc3NlcyA9IFtF bGVtZW50LCBQcm9jZXNzaW5nSW5zdHJ1Y3Rpb24sIENvbW1lbnQsDQogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBUZXh0LCBDREFUQVNlY3Rpb24sIEVudGl0eVJlZmVyZW5jZV0NCkRv Y3VtZW50VHlwZS5jaGlsZENsYXNzZXMgPSBbXQ0KRW50aXR5UmVmZXJlbmNlLmNoaWxkQ2xhc3Nl cyA9IFtFbGVtZW50LCBQcm9jZXNzaW5nSW5zdHJ1Y3Rpb24sIENvbW1lbnQsDQogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIFRleHQsIENEQVRBU2VjdGlvbiwgRW50aXR5UmVmZXJlbmNl XQ0KRWxlbWVudC5jaGlsZENsYXNzZXMgPSBbRWxlbWVudCwgUHJvY2Vzc2luZ0luc3RydWN0aW9u LCBDb21tZW50LA0KICAgICAgICAgICAgICAgICAgICAgICAgVGV4dCwgQ0RBVEFTZWN0aW9uLCBF bnRpdHlSZWZlcmVuY2VdDQpBdHRyLmNoaWxkQ2xhc3NlcyA9IFtUZXh0LCBFbnRpdHlSZWZlcmVu Y2VdDQpQcm9jZXNzaW5nSW5zdHJ1Y3Rpb24uY2hpbGRDbGFzc2VzID0gW10NCkNvbW1lbnQuY2hp bGRDbGFzc2VzID0gW10NClRleHQuY2hpbGRDbGFzc2VzID0gW10NCkNEQVRBU2VjdGlvbi5jaGls ZENsYXNzZXMgPSBbXQ0KRW50aXR5LmNoaWxkQ2xhc3NlcyA9IFtFbGVtZW50LCBQcm9jZXNzaW5n SW5zdHJ1Y3Rpb24sIENvbW1lbnQsDQogICAgICAgICAgICAgICAgICAgICAgIFRleHQsIENEQVRB U2VjdGlvbiwgRW50aXR5UmVmZXJlbmNlXQ0KTm90YXRpb24uY2hpbGRDbGFzc2VzID0gW10NCg0K IyB2aW06dHM9MjphaQ0K --0__=A2kFZT3qDsXrOyFApQpGBoEJxCoVbsRUYNc21gz18D7Arav2w7NYsuZs Content-type: application/octet-stream; name="test.py" Content-Disposition: attachment; filename="test.py" Content-transfer-encoding: base64 IiIiVGhpcyB0ZXN0cyB0aGUgbmV3IFhNTCBET00gcGFja2FnZSBmb3IgY2lyY3VsYXIgcmVmZXJl bmNlIG1lbW9yeSBsZWFrcy4NClRvIHVzZSBpdCwgY29kZSBtdXN0IGJlIGFkZGVkIHRvIGNvcmUu Tm9kZSBhbmQgY29yZS5fbm9kZURhdGEgdG8gaW5jcmVtZW50DQpOb2RlX2NvdW50ZXIgaW4gX19p bml0X18gYW5kIGRlY3JlbWVudCBpdCBpbiBfX2RlbF9fLiIiIg0KDQpmcm9tIHhtbC5kb20gaW1w b3J0IGNvcmV0ZXN0DQpjb3JlID0gY29yZXRlc3QNCg0KZGVmIGNoZWNrTWVtKG1zZyk6DQoJaWYg Y29yZS5Ob2RlLk5vZGVfY291bnRlciBvciBjb3JlLl9ub2RlRGF0YS5Ob2RlX2NvdW50ZXI6DQoJ CWlmIG1zZzogcHJpbnQgJyMjIyMjIyMjIyMjIyMjIyMjIycsbXNnLCcjIyMjIyMjIyMjIyMjIyMj IyMnDQoJCXByaW50ICJFcnJvcjogJWQgTm9kZXMgYW5kICVkIF9ub2RlRGF0YXMgd2VyZSBub3Qg ZGVsZXRlZCIgJSBcDQoJCQkoY29yZS5Ob2RlLk5vZGVfY291bnRlcixjb3JlLl9ub2RlRGF0YS5O b2RlX2NvdW50ZXIpDQoNCmRlZiB0ZXN0MSgpOg0KCXByaW50ICc9PT09PT09PT09PT09PT09PT0g dGVzdDEgPT09PT09PT09PT09PT09PT09Jw0KCWRvYyA9IGNvcmUuY3JlYXRlRG9jdW1lbnQoKQ0K CWRvYyA9IE5vbmUNCgljaGVja01lbSgndGVzdDEgZmFpbGVkJykNCg0KCQ0KZGVmIHRlc3QyKCk6 DQoJcHJpbnQgJz09PT09PT09PT09PT09PT09PSB0ZXN0MiA9PT09PT09PT09PT09PT09PT0nDQoJ ZG9jID0gY29yZS5jcmVhdGVEb2N1bWVudCgpDQoJaHRtbCA9IGRvYy5jcmVhdGVFbGVtZW50KCdo dG1sJykNCiMJYm9keSA9IGRvYy5jcmVhdGVFbGVtZW50KCdib2R5JykNCiMJaHRtbC5pbnNlcnRC ZWZvcmUoYm9keSxOb25lKQ0KIwlodG1sLmluc2VydEJlZm9yZShib2R5LE5vbmUpDQoJZG9jID0g Tm9uZQ0KIwlib2R5ID0gTm9uZQ0KCWh0bWwgPSBOb25lDQoJY2hlY2tNZW0oJ3Rlc3QyIGZhaWxl ZCcpDQoJDQpkZWYgdGVzdDMoKToNCglwcmludCAnPT09PT09PT09PT09PT09PT09IHRlc3QzID09 PT09PT09PT09PT09PT09PScNCglkb2MgPSBjb3JlLmNyZWF0ZURvY3VtZW50KCkNCglodG1sID0g ZG9jLmNyZWF0ZUVsZW1lbnQoJ2h0bWwnKQ0KIwlkb2MuZG9jdW1lbnRFbGVtZW50ID0gaHRtbAkJ IyB0aGlzIGRvZXMgbm90IGR1cGxpY2F0ZSBodG1sIGVsZW1lbnQNCiMJZG9jLmRvY3VtZW50RWxl bWVudCA9IE5vbmUNCglkb2MgPSBOb25lDQoJaHRtbCA9IE5vbmUNCgljaGVja01lbSgndGVzdDMg ZmFpbGVkJykNCg0KZGVmIHRlc3Q0KCk6DQoJcHJpbnQgJz09PT09PT09PT09PT09PT09PSB0ZXN0 NCA9PT09PT09PT09PT09PT09PT0nDQoJZG9jID0gY29yZS5jcmVhdGVEb2N1bWVudCgpDQoJaHRt bCA9IGRvYy5jcmVhdGVFbGVtZW50KCdodG1sJykNCglodG1sLmluc2VydEJlZm9yZShkb2MuY3Jl YXRlRWxlbWVudCgnYm9keScpLE5vbmUpDQoJZG9jLmFwcGVuZENoaWxkKGh0bWwpDQoJZG9jLmlu c2VydEJlZm9yZShodG1sLE5vbmUpDQoJI2RvYy5kb2N1bWVudEVsZW1lbnQgPSBOb25lDQoJZG9j ID0gTm9uZQ0KCWh0bWwgPSBOb25lDQoJY2hlY2tNZW0oJ3Rlc3Q0IGZhaWxlZCcpDQoNCmlmIF9f bmFtZV9fID09ICdfX21haW5fXyc6DQoJdGVzdDEoKQ0KCXRlc3QyKCkNCgl0ZXN0MygpDQoJdGVz dDQoKQ== --0__=A2kFZT3qDsXrOyFApQpGBoEJxCoVbsRUYNc21gz18D7Arav2w7NYsuZs-- From Jeff.Johnson@stn.siemens.com Fri Oct 9 19:55:54 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Fri, 9 Oct 1998 14:55:54 -0400 Subject: [XML-SIG] RE: DOM circular refs Message-ID: <85256698.0067ED64.00@BI01.boca.ssc.siemens.com> Looks like I didn't solve all the problems. Using SaxBuilder to read an XML file, 3 Nodes never get freed and none of the _nodeDatas get freed! I've noticed in the past that if I don't set the DocumentHandler.document to None when finished, it doesn't free, maybe this is a bug somewhere in the builder code. I'll try to find it, one more day won't slip my project too much :) Being a Python newbie, I didn't know about __getattr__ and __setattr__ ten minutes ago. Maybe mucking with those would be a good way of providing a readonly documentElement attribute for Document that raises an exception when set and returns a new proxy when got. Currently I got rid of the attribute in favor of a function call get_documentElement(). What do the python gurus say? Cheers, Jeff From akuchlin@cnri.reston.va.us Fri Oct 9 20:26:25 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Fri, 9 Oct 1998 15:26:25 -0400 (EDT) Subject: [XML-SIG] RE: DOM circular refs In-Reply-To: <85256698.0067ED64.00@BI01.boca.ssc.siemens.com> References: <85256698.0067ED64.00@BI01.boca.ssc.siemens.com> Message-ID: <13854.25286.186919.572177@amarok.cnri.reston.va.us> Jeff.Johnson@stn.siemens.com writes: >Being a Python newbie, I didn't know about __getattr__ and __setattr__ ten >minutes ago. Maybe mucking with those would be a good way of providing a >readonly documentElement attribute for Document that raises an exception >when set and returns a new proxy when got. Currently I got rid of the >attribute in favor of a function call get_documentElement(). What do the >python gurus say? Yes, I've been meaning to write convenience __getattr__ and __setattr__ functions on the Node class that automatically call .get_childNodes(), for example, when you try to access .childNodes. It's a pity that's required, because it'll make accessing .childNodes slower than .get_childNodes() -- you'll call __getattr__ first, which will then call get_childNodes() for you -- but it seems unavoidable. Implementing those two methods are first on my list for tonight. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ The past is only partly irrecoverable. The clerisy should accord it at least as much courtesy as they offer to the future. -- Robertson Davies, _A Voice from the Attic_ From Jeff.Johnson@stn.siemens.com Fri Oct 9 21:28:31 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Fri, 9 Oct 1998 16:28:31 -0400 Subject: [XML-SIG] DOM and xmlproc problem Message-ID: <85256698.0070B42D.00@BI01.boca.ssc.siemens.com> I don't know much about the various XML parsers but I think I have found a circular reference in at least one. When I call the following code... p = saxexts.make_parser() p = None ...the object pointed to by 'p' does not get freed. I know this becuase I modified the __init__ and __del__ functions for class SAX_XPParser in xml.sax.drivers.drv_xmlproc to print "INIT" and "DEL". "DEL" never prints but "INIT" does. I tried to find out why it doesn't get freed but got lost in all the code. I was hoping someone that knows this code would take a look. I don't know if this will affect all users since some people may have different XML parsers than I do. I run in windows and have no idea if pyexpat is being used on my machine, nor do I know how to properly debug Python, I just use a lot of print statements :) DOM: The DOM tree doesn't get freed because the parser points to the builder via "doc_handler", the builder points to the document and to the current_element which also points to the document. To get it to the DOM to free you must either call: builder.document = None builder.current_element = None or call: parser.setDocumentHandler(None) The second method is better because it also frees the builder. That's about as far as I can take it, anyone up for finding the circular reference in the parser? Cheers, Jeff From Fred L. Drake, Jr." References: <13853.13854.985841.118112@weyr.cnri.reston.va.us> <361D3989.16D0856E@lyra.org> Message-ID: <13854.32295.893582.893266@weyr.cnri.reston.va.us> Greg Stein writes: > The DTD wasn't attached :-) Yeah, I realized that as soon as I sent the note! But it went shortly thereafter, so we'll all get over it, I hope. ;-) > Nevertheless, I feel that the metadata element has very little purpose. > Given the presence of XML Namespaces > (http://www.w3.org/TR/WD-xml-names), I don't think that the metadata tag > adds any value. Rather than use , > we can use . Of course, there I tried to explain in the document why namespaces alone are not sufficient, but I guess I wasn't successful. A namespace is really just refers to a structure definition. There is no reason that applications must use their own structure definitions. For example, two might use something that defines an element with that includes a bunch of elements, similar to those in a previous iteration of XBEL: bar nitz You are saying that metadata structure definitions cannot be shared independently of the instance data; does this really make sense? In the presence of generalized things like RDF, I think not. That's why I want to keep another level of owner identification around. > are other ways to specify the use of a namespace, but the point is that > a "metadata-like" facility has already been defined in a standardized > fashion (with the caveat of minor perturbations to the working draft > prior to IETF acceptance). Again, XML namespaces are really meta-information about the structure of the data, and are not related to "ownership", which is much closer to an application-level concept. > %common.attrs% is not very useful as an extension. It is located to > broadly for somebody to extend the DTD. If you try to add something into > %common.attr% it shows up EVERYWHERE. eek. Personally, I'm of the mind Ok, perhaps. I'm used to seeing DTDs that have something that is allowed everywhere, typically an ID to allow addressability and something like HTML's CLASS or DocBook's ROLE. This can then be extended using a local. to add additional attributes. Other groups of attribute definitions then also have their own local. parts that can be defined extended DTDs. Other parameter entities are used to construct content models, etc. How about this: We drop common.att, and add in local.node.att, local.url.add, and local.node.mix. These will be defined to be "", and included at the end of their non-local definition. This removes the more difficult to use common.att, while allowing extensibility using more targeted parameters. > that somebody can issue a second version of the DTD, rather than attempt > to extend it. At the present time, I'm not aware of apps that actually The application would not be the one to extend it. Often, "industry standard" DTDs are subclassed in exactly this way to support additional requirements for internal data/document management. Instances are then down-converted for interchange. This does not strike me as being the "wrong" way. When it doesn't work, it's most likely that what is really needed isn't an extension of the original DTD in the first place. In this environment, it makes a lot of sense for maintainability to use the original public text without change to avoid accidental structural drift during maintenance. > Further, given the presence of XML namespaces, an XBEL document can > simply be incorporated *within* another XML document. i.e. extension via > containment, rather than extension via derivation. As I tried to describe above, the ability to contain isn't the problem; XML handles that quite nicely (esp. with namespaces). Ownership is simply a different problem. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Sat Oct 10 18:04:47 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: 10 Oct 1998 17:04:47 -0000 Subject: [XML-SIG] DOM and xmlproc problem In-Reply-To: <85256698.0070B42D.00@BI01.boca.ssc.siemens.com> Message-ID: <19981010170447.3749.qmail@findmail.com> > I don't know much about the various XML parsers but I think I have found a > circular reference in at least one. When I call the following code... > p = saxexts.make_parser() > p = None > ..the object pointed to by 'p' does not get freed. I've checked in a fix for this; p.parser had some references to some methods of p. I added code to the parser's .close() method which deleted those attributes, so everything's fine as long as you're sure to call p.close() when you've finished parsing. We'll document that as being required. > The DOM tree doesn't get freed because the parser points to the builder via > "doc_handler", the builder points to the document and to the > current_element which also points to the document. I can't seem to reproduce this. Note that what you describe isn't really a cycle; it looks like parser -> builder -> currentelement -> document builder -> document -- amk ----- See the original message at http://www.egroups.com/list/xml-sig/?start=434 -- Free e-mail group hosting at http://www.eGroups.com/ From Jeff.Johnson@stn.siemens.com Mon Oct 12 02:12:28 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Sun, 11 Oct 1998 21:12:28 -0400 Subject: [XML-SIG] Re: XML-SIG digest, Vol 1 #121 - 1 msg Message-ID: <8525669B.00068620.00@BI01.boca.ssc.siemens.com> > I don't know much about the various XML parsers but I think I have found a > circular reference in at least one. When I call the following code... > p = saxexts.make_parser() > p = None > ..the object pointed to by 'p' does not get freed. >From: "Andrew M. Kuchling" >>I've checked in a fix for this; p.parser had some >>references to some methods of p. I added code to the parser's .close() method which deleted those >>attributes, so everything's fine as long as you're >>sure to call p.close() when you've finished >>parsing. We'll document that as being required. Thanks for finding that so fast, I'm so anal retentive I probably would've spent next week trying to find it :) >> The DOM tree doesn't get freed because the parser points to the builder via >> "doc_handler", the builder points to the document and to the >> current_element which also points to the document. >I can't seem to reproduce this. Note that what you describe isn't really a cycle; it looks like >parser -> builder -> currentelement -> document > builder -> document Yeah, I meant the cycle was in the parser and because it didn't free, the DOM didn't free. Now that the parser will free, problem solved :) Thanks again, Jeff From akuchlin@cnri.reston.va.us Wed Oct 14 02:40:25 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Tue, 13 Oct 1998 21:40:25 -0400 Subject: [XML-SIG] DOM: Some open issues Message-ID: <199810140140.VAA00332@mira.erols.com> I'm slowly converging toward having everything implemented (not necessarily having everything bug-free or spec-compliant, mind you). There are some things that the SIG should discuss. * First is the question of DTD representation and creation in the DOM. What should the interface be for creating Notation, Entity, and DocumentType objects: should there be extended createNotation(), ... methods on the Document object? The DOM tree needs to know about the default value of attributes, so that it can restore the default value when you delete the Attr node. Any suggestions for an interface? Could it be as simple as a dictionary on the DocumentType object, where the keys are (elementName, attributeName) tuples and the dict value is the default value? * There are memory leaks in some of the parser drivers. I've fixed the PyExpat driver, but the xmlproc driver defeated me; the leaks might be in xmlproc itself. This will have to be checked... * Minor thing: the exceptions are all named things like IndexSizeException, DOMStringSizeException, HierarchyRequestException, etc. Should I add aliases which drop the 'Exception' from the names? -- A.M. Kuchling http://starship.skyport.net/crew/amk/ As I was going up the stair / I met a man who wasn't there. / He wasn't there again today. / I wish, I wish he'd stay away. -- Hughes Mearns, "The Psychoed" From guido@CNRI.Reston.VA.US Thu Oct 15 15:12:39 1998 From: guido@CNRI.Reston.VA.US (Guido van Rossum) Date: Thu, 15 Oct 1998 10:12:39 -0400 Subject: [XML-SIG] Don't forget the Python Conference! Message-ID: <199810151412.KAA12968@eric.cnri.reston.va.us> Many of you on this list may not realize that the Python conference is less than four weeks away. Don't miss this unique opportunity to meet key players in the Python and XML community, such as Andrew Kuchling, Paul Prescod, and Sean McGrath, as well as many other Python luminaries such as multiple O'Reilly book author Mark Lutz. Also meet Open Source guru Eric Raymond, who will give a keynote speech. If you're just beginning to use Python and XML, don't miss the tutorial day: an ideal combination might be the intro to Python (by Robin Friedrich) in the morning, and the XML tutorial (by Sean McGrath and Paul Prescod) in the afternoon. Register and pay by October 16 (tomorrow!) to save $75 on registration. For more info, see: http://www.foretec.com/python/workshops/1998-11/ The four-day conference starts on Tuesday, November 10, in Houston, Texas. --Guido van Rossum (home page: http://www.python.org/~guido/) From akuchlin@cnri.reston.va.us Thu Oct 15 15:15:30 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Thu, 15 Oct 1998 10:15:30 -0400 (EDT) Subject: [XML-SIG] Anything else to go in? Message-ID: <199810151415.KAA19697@amarok.cnri.reston.va.us> Question for discussion: The Python/XML package currently contains SAX and DOM interfaces, plus two parsers, plus xmlarch, plus some miscellaneous things. Is there anything else which needs to go into the package before 1.0? http://www.python.org/sigs/xml-sig/status.html lists the following things: * The glue code required to use existing Java XML parsers from JPython, and the appropriate documentation. * An XLL implementation that navigates a DOM tree (LMG's XPointer parser is part of this.) * A module to marshal simple Python data types into XML. There's still no obvious DTD to choose for this, though; I'm starting to think that I should drop xml/marshal.py, and wait until version 1.1 of the package to add this; perhaps by then one DTD will have emerged as the standard for doing this. If no one suggests anything else which needs to be added, then 1.0 will look essentially like the CVS tree + bugfixes + additional docs, demos and test code. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Give light, and the darkness will disappear of itself. -- Desiderius Erasmus From ken@bitsko.slc.ut.us Thu Oct 15 19:40:56 1998 From: ken@bitsko.slc.ut.us (Ken MacLeod) Date: Thu, 15 Oct 1998 13:40:56 -0500 (CDT) Subject: [XML-SIG] Anything else to go in? In-Reply-To: <199810151415.KAA19697@amarok.cnri.reston.va.us> from "Andrew M. Kuchling" at Oct 15, 98 10:15:30 am Message-ID: <199810151840.NAA22921@bitsko.slc.ut.us> > * A module to marshal simple Python data types into XML. > There's still no obvious DTD to choose for this, though; I'm starting > to think that I should drop xml/marshal.py, and wait until version 1.1 > of the package to add this; perhaps by then one DTD will have emerged > as the standard for doing this. I've just completed a draft DTD for Casbah/LDO XML serialization (below). This DTD can be targeted, so you can be as specific about Python types as pickle, or interoperable as with Casbah/LDO. Note specifically that any object can be used as a key in a dictionary, as Python (and SmallTalk, for example) support. Most other DTDs I've seen only support string keys. As an implementation note, LDO's Python binary serialization uses pickle's `dump' and `load' methods, it also can act as a stream-head so it supports `flush' as well. The source is in CVS at: CVSROOT=:pserver:anonymous@ntlug.org:/home/cbsrc/cvsroot password: anonymous module: LDO or viewable at . This is meant to be an open spec, please feel free to comment on it and make suggestions, either here, on the Casbah list, or to me. Thanks, -- Ken -------- cut here -------- From grove@infotek.no Fri Oct 16 15:07:31 1998 From: grove@infotek.no (Geir Ove Gronmo) Date: Fri, 16 Oct 1998 16:07:31 +0200 Subject: [XML-SIG] xmlarch: version 0.20 released Message-ID: <199810161407.QAA25512@mail.infotek.no> xmlarch: An XML architectural forms processor written in Python Version: 0.20 Released: October 16th 1998 Author: Geir Ove Gr�nmo Email: grove@infotek.no Homepage: http://www.infotek.no/~grove/software/xmlarch/index.html --- What is xmlarch? The xmlarch.py module contains an XML architectural forms processor written in Python. It allows you to process XML architectural forms using any parser that uses the SAX interfaces. The module allow you to process several architectures in one parse-pass. Architectural document events for an architecture can even be broadcasted to multiple DocumentHandlers. What's new? - The module is better documented. Class reference documentation has been generated (pythondoc) and is included in the distribution. - Added full support for renaming. This includes support for #CONTENT, #ARCCONT, #DEFAULT and #MAPTOKEN. - Added the possibility to get information about the original events (EventTracker). This means e.g. that it is possible to find out what element was used in the client document. A more complete and higher level api will be provided later. - Normalizer has been renamed to Prettifier. It is now more complete and generates "prettified" output. Fixes: - Suppressor attributes should apply to the element's descendants, not itself. - Some minor ones --- Enjoy! Geir Ove Gr�nmo From ken@bitsko.slc.ut.us Fri Oct 16 19:52:11 1998 From: ken@bitsko.slc.ut.us (Ken MacLeod) Date: Fri, 16 Oct 1998 13:52:11 -0500 (CDT) Subject: [XML-SIG] Anything else to go in? In-Reply-To: <199810151840.NAA22921@bitsko.slc.ut.us> from "Ken MacLeod" at Oct 15, 98 01:40:56 pm Message-ID: <199810161852.NAA23634@bitsko.slc.ut.us> Earlier I wrote: > I've just completed a draft DTD for Casbah/LDO XML serialization (below). > This DTD can be targeted, so you can be as specific about Python types > as pickle, or interoperable as with Casbah/LDO. Just a quick followup, I got some feedback mentioning that the element was misleading (is it the value of a key-value pair, etc.). I've changed to to reflect that this item is an atomic or literal value. It also wasn't clearly stated that internal non-string values (like numbers) should be converted to their natural string equivalents when marshaled to an . -- Ken From akuchlin@cnri.reston.va.us Fri Oct 16 22:37:48 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Fri, 16 Oct 1998 17:37:48 -0400 (EDT) Subject: [XML-SIG] Anything else to go in? In-Reply-To: <199810151840.NAA22921@bitsko.slc.ut.us> References: <199810151415.KAA19697@amarok.cnri.reston.va.us> <199810151840.NAA22921@bitsko.slc.ut.us> Message-ID: <13863.47958.180567.334245@amarok.cnri.reston.va.us> Ken MacLeod writes: > ... > `ref' attribute of the `ref' element. `dictionary', `list' and > `value' elements support a `type' attribute to declare the type > or class of the object. `value' elements have an `encoding' This DTD looks simple to implement, requiring just a bunch of changes to the existing marshal.py. However, I'd like to make the data structure -> LDO -> data structure transformation not lose any information, so I'd like to agree on certain coventional values for the 'type' attribute, such as integer, float, 'tuple' for the list element, etc. Do you think it's possible to nail that down now, or is it premature? -- A.M. Kuchling http://starship.skyport.net/crew/amk/ This is, after all, a book about reading, and the kind of reader I am addressing does not care primarily about being in fashion. -- Robertson Davies, _A Voice from the Attic_ From ken@bitsko.slc.ut.us Fri Oct 16 23:18:14 1998 From: ken@bitsko.slc.ut.us (Ken MacLeod) Date: Fri, 16 Oct 1998 17:18:14 -0500 (CDT) Subject: [XML-SIG] Anything else to go in? In-Reply-To: <13863.47958.180567.334245@amarok.cnri.reston.va.us> from "Andrew M. Kuchling" at Oct 16, 98 05:37:48 pm Message-ID: <199810162218.RAA28348@bitsko.slc.ut.us> Andrew Kuchling writes: > Ken MacLeod writes: > > > ... > > `ref' attribute of the `ref' element. `dictionary', `list' > > and `value' elements support a `type' attribute to declare the > > type or class of the object. `value' elements have an > > `encoding' > > This DTD looks simple to implement, requiring just a bunch of > changes to the existing marshal.py. However, I'd like to make the > data structure -> LDO -> data structure transformation not lose any > information, so I'd like to agree on certain coventional values for > the 'type' attribute, such as integer, float, 'tuple' for the list > element, etc. Do you think it's possible to nail that down now, or > is it premature? Yes, I think now's a good time. The DTD isn't set up to be formally specialized, but it was intended to be informally specialized, which is what we'd be doing by nailing down the types that Python would use for the transformation. (Formal specialization would use XML stuff like external DTDs and parameter entities, similar to the DocBook SGML DTD.) The general pattern I've had in mind for specialization is to start off by just using the `type' attribute where appropriate to explicitly declare what the element is supposed to be. If simply giving a `type' doesn't fit exactly, then a more complex ``encoding'' would be used, where some rules are applied to translate the internal object type to a suitable element with a certain `type' attribute. As an FYI, the Python LDOConnection module should be able to use both a Python-specialized marshaler to give a full-featured Python-to-Python remote objects or procedure calls, as well as a generic marshaler to give Python-to-other calls (as in other LDO implementations). In a sense, the binary implementation already has this feature because you can swap-in(*) pickle.py for LDOBinary.py. (* in theory -- the code to actually do it isn't there yet.) The LDOBinary module was implemented as much by empirical testing as by reading pickle.py and others, so I'm not sure how much I can help with a truly lossless implementation :-). I have a good feel for the pattern I described above so I can help make sure what we come up with is consistent with that. Let me know what you'd like me to do. -- Ken From akuchlin@cnri.reston.va.us Sat Oct 17 00:08:39 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Fri, 16 Oct 1998 19:08:39 -0400 (EDT) Subject: [XML-SIG] Anything else to go in? In-Reply-To: <199810162218.RAA28348@bitsko.slc.ut.us> References: <13863.47958.180567.334245@amarok.cnri.reston.va.us> <199810162218.RAA28348@bitsko.slc.ut.us> Message-ID: <13863.53339.497854.772699@amarok.cnri.reston.va.us> Ken MacLeod writes: >The LDOBinary module was implemented as much by empirical testing as >by reading pickle.py and others, so I'm not sure how much I can help >with a truly lossless implementation :-). I have a good feel for the >pattern I described above so I can help make sure what we come up with >is consistent with that. Let me know what you'd like me to do. Well, the simple types such as integers and floats are easy. They're fairly similar across languages; one might worry about different integer sizes and FP precisions, but the unmarshalling code might simply implement a "best effort". For example, on a 64-bit platform you might generate 18014398509481984, which won't fit in a 32-bit integer; a 32-bit platform confronted with this would have to deal with it by converting to a long or BigInteger or whatever. I'll proceed for now with "integer", "long", "float", "complex", "string", and see how that goes, but perhaps the type attribute should have values like i32/i64, instead, and be more precise about the limits. Objects are a whole other kettle of fish. In Python they could basically be dictionaries mapping from attribute -> value. But how would one specify the class of the object? In Python there's an attribute called __class__, but that shouldn't creep into LDO. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Errors using inadequate data are much less than those using no data at all. -- Charles Babbage From ken@bitsko.slc.ut.us Wed Oct 14 17:11:31 1998 From: ken@bitsko.slc.ut.us (Ken MacLeod) Date: 14 Oct 1998 11:11:31 -0500 Subject: [XML-SIG] Anything else to go in? In-Reply-To: "Andrew M. Kuchling"'s message of Fri, 16 Oct 1998 19:08:39 -0400 (EDT) References: <13863.47958.180567.334245@amarok.cnri.reston.va.us> <199810162218.RAA28348@bitsko.slc.ut.us> <13863.53339.497854.772699@amarok.cnri.reston.va.us> Message-ID: "Andrew M. Kuchling" writes: > Ken MacLeod writes: > >The LDOBinary module was implemented as much by empirical testing > >as by reading pickle.py and others, so I'm not sure how much I can > >help with a truly lossless implementation :-). I have a good feel > >for the pattern I described above so I can help make sure what we > >come up with is consistent with that. Let me know what you'd like > >me to do. > > Well, the simple types such as integers and floats are easy. > They're fairly similar across languages; one might worry about > different integer sizes and FP precisions, but the unmarshalling > code might simply implement a "best effort". For example, on a > 64-bit platform you might generate type="integer">18014398509481984, which won't fit in a > 32-bit integer; a 32-bit platform confronted with this would have to > deal with it by converting to a long or BigInteger or whatever. > I'll proceed for now with "integer", "long", "float", "complex", > "string", and see how that goes, but perhaps the type attribute > should have values like i32/i64, instead, and be more precise about > the limits. Ah, I thought you were talking about a more Python-specific implementation. For LDO, you've run into one of our more interesting design issues, so I welcome more people thinking about it. LDO is geared towards the scripting, CGI, property list, untyped container level -- in other words, where strings, numerics, and booleans are distinguished by usage not by predeclared type. Even the more complex objects are typed more by convention than by strict adherence to an object structure. In general, most of the languages we've surveyed handle this well, they either coerce primitive types by usage or they have explicit interfaces that can be queried to force coercion. Python is one of the languages that doesn't fit either category. I first recognized this while writing LDOBinary: Python overloads string operations onto numeric operators. We discussed this on the Casbah mail list and irc and came to the preliminary conclusion that we would like to slide Python into the latter category, where Python clients and servers will need wrappers or IDLs to talk to non-Python peers. One specific item that is important: the question is not whether we can add type information to the format (we can) the question is whether storage back-ends and other languages have or carry the type information necessary to be encoded (several don't). This is a very deep issue and I don't want to make it sound like it's a done deal, our biggest need is for more information and I don't mind at all keeping the issue open. Since we would prefer to keep LDO on the typeless end, we are currently continuing down that path. About type sizes, yes it is the receiver's responsibility to check for overflow. For explicitly typed languages, the size is known, for implicitly typed languages the largest type necessary should be used, Python's BigInteger definitely qualifies here. > Objects are a whole other kettle of fish. In Python they > could basically be dictionaries mapping from attribute -> value. > But how would one specify the class of the object? In Python > there's an attribute called __class__, but that shouldn't creep into > LDO. On the brighter side :-), when it comes to ``objects'' (a.k.a. property lists, records, entries, etc.) the types or classes are whatever the creator or owner says they are. In LDO, it is the client's responsibility to encode objects with either the server's desired type/class name or an agreed-upon type/class name. From the server side, feel free to copy Python's __class__ to the LDO XML `type' attributed. The pattern I've thought of using for the client end is to have a per-connection type-mapping delegate. Each client-side LDOConnection will have a type-mapping delegate that it will provide for the marshaler (delegates can be share between connections). The marshaler can then call the delegate with the local type and it will return the server's required type, or vice-versa. The client-side type doesn't even necessarily have to be a valid local class, as long as it gets carried with the object and can be handed back to the server as necessary. The delegate may be a possible solution to the type issue above, handling type-conversion as specified by configuration or IDL. The delegate may also be a good place to interface to a Python object's __getstate__ and __setstate__ methods. -- Ken MacLeod ken@bitsko.slc.ut.us From uche.ogbuji@fourthought.com Sun Oct 18 18:45:20 1998 From: uche.ogbuji@fourthought.com (uche.ogbuji@fourthought.com) Date: Sun, 18 Oct 1998 10:45:20 -0700 Subject: [XML-SIG] Patch for xmlproc: proper SAX error-handling in validating parsers Message-ID: <199810181745.KAA02835@malatesta.local> This is a multipart MIME message. --==_Exmh_-16799825460 Content-Type: text/plain; charset=us-ascii I know that Lars is off for the month, and I've been investigating a problem where the SAX driver for the validating xmlproc parser was not calling the appropriate SAX driver functions for error handling: It was instead using the xmlproc API report_error method, which simply prints the error to stdout. I needed a proper SAXException or SAXParseException generated and passed to the SAX ErrorHandler error or fatalError method. I found what appears to be the problem, and I have a patch (attached) for xml.sax.drivers.drv_xmlproc_val.py that fixes it. I hope that until Lars returns a) It helps someone else having the same problems with SAX/xmlproc error reporting as I had. b) Others can test and make sure it works for them and doesn't break other things. Thanks. -- Uche Ogbuji uche.ogbuji@fourthought.com Consulting Member, FourThought LLC http://FourThought.com http://OpenTechnology.org (970)481-0805 --==_Exmh_-16799825460 Content-Type: text/plain; name="valdrv_error_patch.diff"; charset=us-ascii Content-Description: valdrv_error_patch.diff Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="valdrv_error_patch.diff" *** drv_xmlproc_val.py Sun Oct 18 10:42:16 1998 --- uche/drv_xmlproc_val.py Sun Oct 18 02:40:08 1998 *************** *** 5,11 **** version=3D"0.91" = from xml.sax import saxlib,saxutils ! from xml.parsers.xmlproc import xmlval from xml.sax.drivers.drv_xmlproc import * = import types --- 5,11 ---- version=3D"0.91" = from xml.sax import saxlib,saxutils ! from xml.parsers.xmlproc import xmlval, errors from xml.sax.drivers.drv_xmlproc import * = import types *************** *** 19,27 **** self.parser.set_error_handler(self) self.dtd=3Dself.parser.get_dtd() self.doc_handler.setDocumentLocator(self) ! = def _create_parser(self): ! return xmlval.XMLValidator() = def handle_start_tag(self, name, attrs): try: --- 19,31 ---- self.parser.set_error_handler(self) self.dtd=3Dself.parser.get_dtd() self.doc_handler.setDocumentLocator(self) ! #needed to placate xmlval.XMLValidator.handle_doctype = ! self.errors =3D errors.english ! = def _create_parser(self): ! validator =3D xmlval.XMLValidator() ! validator.val.parser =3D self ! return validator = def handle_start_tag(self, name, attrs): try: *************** *** 30,35 **** --- 34,43 ---- self.dtd.get_ele= m(name))) except KeyError,e: self.doc_handler.startElement(name,XPAttributes(attrs,None)= ) + = + def report_error(self, msg, param): + message =3D (self.errors[msg])%(param) + self.error(message) = # --- EXPERIMENTAL PYTHON SAX EXTENSIONS: = --==_Exmh_-16799825460-- From Fred L. Drake, Jr." I've posted a new draft of the XBEL documentation; it's at http://www.python.org/sigs/xml-sig/xbel/doc/xbel.html I think it is complete, but the "bibliography" still needs work and citation marks need to be added to the text. Note that in the DTD, I made the "owner" attribute for #REQUIRED, to match the documentation and the intended semantics as discussed here on the list. I've sent this version to Andrew for incorporation into the CVS repository. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Mon Oct 19 22:30:17 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Mon, 19 Oct 1998 17:30:17 -0400 (EDT) Subject: [XML-SIG] Status report Message-ID: <13867.44412.442335.443002@amarok.cnri.reston.va.us> The rate of hacking on the DOM code itself is now going to drop a lot; I'm going to work on documenting it, and on updating some of the demo programs. Of course, bugs will continue to be fixed as they get reported; my thanks to Jeff Johnson, who's been the most serious user of the DOM code so far, and has reported all sorts of bugs. There are still unimplemented bits, mostly in classes related to the DTD structure, such as Notation and Entity. I'm not going to work on these until I get some guidance from the SIG on how to proceed. These bits are going to be low-priority for me until some documentation has been written for the basic interfaces. So, please pick up the DOM code and start using it, in order to shake more bugs out of it. Once documentation has been written, I intend to produce a 0.5 release and announce it, so finding bugs before that release would be very helpful. (Otherwise, we'll find them after the release...) -- A.M. Kuchling http://starship.skyport.net/crew/amk/ To be pleased with one's limits is a wretched state. -- Goethe (1791) Underachiever -- and proud of it, man! -- The Simpsons (1991) From Jeff.Johnson@stn.siemens.com Tue Oct 20 15:16:05 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Tue, 20 Oct 1998 10:16:05 -0400 Subject: [XML-SIG] pyexpat on Windows? Message-ID: <852566A3.004E485F.00@BI01.boca.ssc.siemens.com> Jeff Johnson writes: > As you may have seen in my emails with Andrew K., the xmlproc parser has a > memory leak of some sort. On my Windows machines, xmlproc gets used by > default and if I try to use pyexpat, I get the following: > > C:\Python\xml\test>python test_pyexpat.py > Traceback (innermost last): > File "test_pyexpat.py", line 11, in ? > import pyexpat > ImportError: No module named pyexpat > > Could you tell me what I need to do to get this setup on Windows? Jack Jansen writes: >>Well, the first step is to get it compiled, of course, and since I don't use >>Windows regularly I haven't included a makefile or project or whatever for >>that. Either create one yourself (and please send me the result) or ask on the >>XML sig whether someone else has done this. After that running the test should >>work fine if you're running in the directory where pyexpat.pyd is (and where >>the test program also resides), but eventually it's probably a good idea to >>put the .pyd somewhere on your sys.path. Either add the directory where >>pyexpat.pyd lives to your path initialization or move the pyd file to a >>directory that is already on the path. I'm still a python newbie and have a never compiled a pyd and my C compiler has dust on it. I see that there is a file xml\windows\pyexpat.dll in the XML package. Is it sufficient to rename this to pyexpat.pyd and move it somewhere? If pyexpat isn't compiled for Windows yet, what is pyexpat.dll? If it is, why isn't it set up to work in the XML package? What will it take to set it up? Does anyone know which parser is faster/better? Thanks, Jeff From akuchlin@cnri.reston.va.us Tue Oct 20 16:43:22 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 20 Oct 1998 11:43:22 -0400 (EDT) Subject: [XML-SIG] pyexpat on Windows? In-Reply-To: <852566A3.004E485F.00@BI01.boca.ssc.siemens.com> References: <852566A3.004E485F.00@BI01.boca.ssc.siemens.com> Message-ID: <13868.44176.915972.391125@amarok.cnri.reston.va.us> Jeff.Johnson@stn.siemens.com writes: >I'm still a python newbie and have a never compiled a pyd and my C compiler >has dust on it. I see that there is a file xml\windows\pyexpat.dll in the >XML package. Is it sufficient to rename this to pyexpat.pyd and move it >somewhere? If pyexpat isn't compiled for Windows yet, what is pyexpat.dll? >If it is, why isn't it set up to work in the XML package? What will it >take to set it up? Does anyone know which parser is faster/better? Sean McGrath contributed the Windows-compiled versions of sgmlop.c and pyexpat.c. I can't test them, and don't know how to set them up on Windows; help with this would be *greatly* appreciated. Since Windows users may not have make or other compilation tools installed, perhaps the best solution would be to have a .ZIP file containing the .py files, pre-compiled .pyd files, etc. that could simply be unpacked in the right directory to install it. Fred Drake has suggested moving sgmlop.so and pyexpat.so on Unix platforms into xml.parser, since Python can handle C extensions inside a package. I assume this would also work on Windows and Mac platforms; am I correct? If so, I'll make this change to the CVS tree tonight. Jack Jansen has also produced a new version of pyexpat, which fixes the exception-swallowing bug and some other things; I'll check that in tonight, too. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Now. Take this street to New Mexico. -- ??? in DOOM PATROL #50 From digitome@iol.ie Tue Oct 20 16:39:55 1998 From: digitome@iol.ie (Sean Mc Grath) Date: Tue, 20 Oct 1998 16:39:55 +0100 Subject: [XML-SIG] pyexpat on Windows? In-Reply-To: <852566A3.004E485F.00@BI01.boca.ssc.siemens.com> Message-ID: <3.0.6.32.19981020163955.00925380@gpo.iol.ie> [Jeff Johnson] > If pyexpat isn't compiled for Windows yet, what is pyexpat.dll? It is indeed pyexpat for windows. I compiled it. It is still being tested but it works fine for me here. Sean Mc Grath def Get_URI_Of_Superlative_Scripting_Language(): return "http://www.python.org" From Jeff.Johnson@stn.siemens.com Tue Oct 20 18:16:33 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Tue, 20 Oct 1998 13:16:33 -0400 Subject: [XML-SIG] pyexpat on windows Message-ID: <852566A3.005EA33D.00@BI01.boca.ssc.siemens.com> Thanks for the thanks Andrew :) It made my day. BTW, I processed 13,753 html files in one process last night and didn't run out of memory! Thanks for re-writing the XML package. Sean Mc Grath writes: >It is indeed pyexpat for windows. I compiled it. It is still being >tested but it works fine for me here. I tried copying pyexpat.dll to \python\DLLs and got an error (see below). Could you tell us what to do with pyexpat.dll so that it will work in Windows? Thanks. C:\Python\xml\test>python test_pyexpat.py Traceback (innermost last): File "test_pyexpat.py", line 11, in ? import pyexpat ImportError: DLL load failed: A device attached to the system is not functioning. From akuchlin@cnri.reston.va.us Tue Oct 20 18:24:58 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 20 Oct 1998 13:24:58 -0400 (EDT) Subject: [XML-SIG] pyexpat on windows In-Reply-To: <852566A3.005EA33D.00@BI01.boca.ssc.siemens.com> References: <852566A3.005EA33D.00@BI01.boca.ssc.siemens.com> Message-ID: <13868.50928.974950.187197@amarok.cnri.reston.va.us> Jeff.Johnson@stn.siemens.com writes: >I tried copying pyexpat.dll to \python\DLLs and got an error (see below). >Could you tell us what to do with pyexpat.dll so that it will work in >Windows? Thanks. It's possible that the DLL has been corrupted by being checked into the CVS tree; I thought I committed it in binary mode, but could be wrong about that. Sean, you may want to try downloading the tree through anonymous CVS and comparing the binary that you get; let me know if the DLL is being corrupted, and I'll fix it. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ It is lost, lovely child, somewhere in the ragbag that I laughingly refer to as my memory. -- Robertson Davies, "A Conversation about _Dr. Canon's Cure_" From akuchlin@cnri.reston.va.us Tue Oct 20 19:07:27 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 20 Oct 1998 14:07:27 -0400 (EDT) Subject: [XML-SIG] pyexpat on windows In-Reply-To: <3.0.6.32.19981020190002.0090dc80@gpo.iol.ie> References: <852566A3.005EA33D.00@BI01.boca.ssc.siemens.com> <13868.50928.974950.187197@amarok.cnri.reston.va.us> <3.0.6.32.19981020190002.0090dc80@gpo.iol.ie> Message-ID: <13868.53518.429774.429985@amarok.cnri.reston.va.us> Sean Mc Grath writes: >Yikes! Yes indeed. The CVS one is corrupt. Got it; I'll fix that tonight! -- A.M. Kuchling http://starship.skyport.net/crew/amk/ ... Sir Isaac Newton... is in every Englishman's wallet... he's on the English one-pound note. I always carry one on me for good luck. A man who discovered gravity and thus successfully secured our feet on the ground is a good companion. -- Peter Greenaway, _The Belly of an Architect_ From Fred L. Drake, Jr." I fixed a few small nits and updated the copy in the XML-SIG area on python.org. I think the XBEL documentation is almost complete. I'd like everyone who contributed to check their name in the documentation and let me know if you want it presented differently (add/remove initials, whatever), or if I have the wrong characters (esp. for European names). The HTML version is at http://www.python.org/sigs/xml-sig/xbel/doc/xbel.html and a PostScript version is available as well: http://www.python.org/sigs/xml-sig/xbel/xbel.ps I'd like to make a general announcement of XBEL to the larger XML community early next week if there are no objections or further changes. I'll plan on sending the announcement to xml-dev, comp.text.xml, and the tkbrowser-sig. Are there any other forums which might be interested in XBEL? Perhaps a Mozilla list? -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Wed Oct 21 00:29:16 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Tue, 20 Oct 1998 19:29:16 -0400 (EDT) Subject: [XML-SIG] DOM: Whitespace, and subclassing Message-ID: <199810202329.TAA28759@amarok.cnri.reston.va.us> Two questions have come to my mind, after a discussion with Greg Ward, a co-worker who's learning the DOM. * First, for many (but not all) DTDs, you know that Text nodes containing only whitespace can be ignored. For example, in: Text The whitespace between and , and and , may be of no interest. In this case, you'd write a function that walked the tree and deleted any Text nodes that contain only whitespace. (Greg points out you should call .normalize() on the root element first, in case the tree was constructed with only single-character Text nodes. :) ) Question 1: Does such a lose-the-whitespace method seem generally useful enough to be added to the package? 2) If so, should it be added to the core, or put in a xml.dom.util module? 3) Other whitespace normalizations are possible, such as dropping leading and/or trailing whitespace on Text nodes, or shortening runs of whitespace characters down to a single character. Should these be made available? Anyone care to suggest an interface? Second notion: subclassing core DOM classes. Right now, there's no point in subclassing a DOM class such as Element, because the only official way to get an Element node is to call the .createElement() factory method on a Document object, and .createElement() always instantiates an instance of Element. You could never instantiate your subclass! Question: Does subclassing basic DOM classes seem useful for some purpose? If so, how could it be made possible? Perhaps .create*() could take an optional klass = argument, and verify that klass is a subclass of the original class. However, inside core.py, nodes are often created directly, without calling the Document object's .create*() method, in order to save a method call, and that code would have to be changed to use the Document object's factory functions. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Only the phoenix arises and does not descend. And everything changes. And nothing is truly lost. -- The true end of the series, in SANDMAN #74, "The Exile" From Fred L. Drake, Jr." References: <199810202329.TAA28759@amarok.cnri.reston.va.us> Message-ID: <13869.9211.120082.319837@weyr.cnri.reston.va.us> Andrew M. Kuchling writes: > Question 1: Does such a lose-the-whitespace method seem > generally useful enough to be added to the package? > > 2) If so, should it be added to the core, or put in a > xml.dom.util module? Since this is, as you point out, dependent on an application's knowledge of the use of the data, it should be separate. The use of such processing is highly dependent on application knowledge and the availability of a validating parser. Using a validating parser would allow more general knowledge to be applied and less application specific interpretation would be needed. In either case (validating or non-validating parser), the notion of data normalization (as opposed to DOM's node-based normalization) should be distinct from the DOM implementation. > 3) Other whitespace normalizations are possible, such as > dropping leading and/or trailing whitespace on Text nodes, or > shortening runs of whitespace characters down to a single character. > Should these be made available? Anyone care to suggest an interface? Yes, they should be available, as long as they're clearly documented and separate from the actual DOM implementation. They should be destructive on the DOM tree; if you want to work on a copy, make a copy. Otherwise, I don't know what the interface should be. I think a simple function that accepts a two required parameters: the Document object and a node to start from (to allow operating on a subtree). Additional parameters would be specific to the transformation. > Question: Does subclassing basic DOM classes seem useful for > some purpose? If so, how could it be made possible? > > Perhaps .create*() could take an optional klass = object> argument, and verify that klass is a subclass of the original > class. However, inside core.py, nodes are often created directly, > without calling the Document object's .create*() method, in order to > save a method call, and that code would have to be changed to use the > Document object's factory functions. This is a real trade-off problem; performance is generally an issue with some of the XML stuff I've been doing (because the user is waiting for a dialog). Whether the use of subclassed DOM objects requires the use of factories depends on how the framework operates. If you want to be able to "install" a subclass as the default for a particular purpose, maybe. (Not really, this is Python, but it would help for maintainability.) If you only want to be able to create the subclassed variants on demand, the problem pretty much gets shoved off to the builder. Perhaps some explanation of the motivation would help make the value of this clear. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Wed Oct 21 03:50:10 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Tue, 20 Oct 1998 22:50:10 -0400 Subject: [XML-SIG] Latest CVS checkins Message-ID: <199810210250.WAA04621@mira.erols.com> As of Tuesday evening, I've checked in some more changes to the DOM implementation; NodeLists and NamedNodeMaps should now be live, the way they're supposed to be. The complicated special case of .getElementsByTagName(), mentioned in my earlier message, has not been fixed. The latest pyexpat.c has been added. I've tried to check in pyexpat.dll and sgmlop.dll in binary mode; please let me know if they're now OK. Also, a .dump() method has been added to the Node class for debugging purposes, which prints an indented tree of the repr() of each node. If someone can suggest a better name for the method, I'll change it; otherwise, it'll stay as .dump(). Sample output: Useful for figuring out what's going on... -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Larger projects generally do not meet one or more of the criteria for success: schedule, budget, or customer satisfaction. Furthermore, a post-mortem of the large project will not pinpoint the explicit causes of failure. -- J.D. Aron, _The Program Development Process: The Programming Team_ (1983) From akuchlin@cnri.reston.va.us Wed Oct 21 03:16:13 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Tue, 20 Oct 1998 22:16:13 -0400 Subject: [XML-SIG] DOM: .getElementsByTagName problem! Message-ID: <199810210216.WAA04469@mira.erols.com> I was working on the NodeList and NamedNodeList classes tonight, and realized that the DOM spec says that the NodeList returned from getElementsByTagName() is live; its contents change to reflect changes in the underlying document! .getElementsByTagName(tagName) returns a NodeList, which has essentially the semantics of a Python list, contains all Element nodes named tagName, in the order they'd be encountered in a pre-order traversal of the tree. Making this live means that, when you rearrange the tree, any affected NodeLists are magically updated. Implementing this seems nightmarishly difficult; any structural changes to the document require updating those NodeLists, and I can't see a much better implementation than just retraversing the whole tree. Worse, I think this will end up re-introducing cycles and memory leaks. Here's my current, quickly formed, idea of how to handle this. * _nodeData instances need to have a dictionary on them; call this dictionary _D. _D[ elementName ] is a list containing the _nodeData instances that are returned by .getElementsByTagName( elementName ). * The NodeList instance would then hold a reference to the proper list, and if the underlying list gets mutated, the NodeList picks up the change. The NodeList also probably needs to hold a reference to the _nodeData instance, because you need to garbage-collect the entry in _D when the NodeList gets deleted. (How to handle multiple calls for the same element and tag name, though?) * When the tree is modified by inserting a new Element node or subtree, you need to walk from the modified node back up to the Document Node. (I don't know how to do that; remember, the whole point of using _nodeData instances was to avoid keeping pointers to the parent node.) Along the way, when a node is found to have a non-empty _D, you need to regenerate all the lists, in case they've changed. This would require one complete traversal of the subtree below the node (you'd try to do all the NodeLists in one traversal.) I have a bad feeling about this plan; it's complicated, it adds nasty garbage-collection complexities, and I suspect it'll re-introduce complicated cycles. Does anyone have a better idea? Help! (Aarrgh! why the hell did they put such database-style complexity in the Level 1 spec? Who expects the results of .getElementsByTagName to be updated, anyway? I certainly didn't...) -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Something Dian used to say to me. She'd say, "Wes, don't say anything unless you've got something to say." Advice I took to heart. She would also say, "It's a long, long trail that has no turning." And how right she was. -- Wesley Dodds, in SANDMAN #72, part three of "The Wake" From Jack.Jansen@cwi.nl Wed Oct 21 10:17:08 1998 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Wed, 21 Oct 1998 11:17:08 +0200 Subject: [XML-SIG] DOM: Whitespace, and subclassing In-Reply-To: Message by "Andrew M. Kuchling" , Tue, 20 Oct 1998 19:29:16 -0400 (EDT) , <199810202329.TAA28759@amarok.cnri.reston.va.us> Message-ID: > Question 1: Does such a lose-the-whitespace method seem > generally useful enough to be added to the package? I would prefer a wrapper around dom that simply wouldn't return the whitespace, in stead of actually stripping it (what it would do with inserts, whether they'd be placed before are after non-returned-whitespace, is open to discussion). If this is the idiom of choice then a program that reads an xml document, modifies it and writes it out again will probably leave comments and the majority of formatting intact. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@cwi.nl | ++++ if you agree copy these lines to your sig ++++ http://www.cwi.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm From uche.ogbuji@fourthought.com Fri Oct 23 02:55:26 1998 From: uche.ogbuji@fourthought.com (uche.ogbuji@fourthought.com) Date: Thu, 22 Oct 1998 18:55:26 -0700 Subject: [XML-SIG] Another patch for saxlib/xmlproc Message-ID: <199810230155.SAA31031@malatesta.local> This is a multipart MIME message. --==_Exmh_-3968228810 Content-Type: text/plain; charset=us-ascii I was having another problem with saxlib/xmlproc: The DOCTYPE declaration of a parsed document was not being passed along to the registered entity resolver. This is the behavior of JP Clark's XP, and makes sense, as it would be the only applicable event interface for processing the DOCTYPE declaration. After some work, I've come up with a patch that fixes this for me, until Lars is back on-line. Here it is in case anyone else has had the same problem. -- Uche Ogbuji uche.ogbuji@fourthought.com Consulting Member, FourThought LLC http://FourThought.com http://OpenTechnology.org (970)481-0805 --==_Exmh_-3968228810 Content-Type: text/plain; name="entity_resolve_doctype.diff"; charset=us-ascii Content-Description: entity_resolve_doctype.diff Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="entity_resolve_doctype.diff" *** drv_xmlproc.py Thu Oct 22 18:37:09 1998 --- uche/drv_xmlproc.py Thu Oct 22 18:43:34 1998 *************** *** 108,113 **** --- 108,118 ---- else:=0D return newsysid=0D =0D + #Make sure that the DOCTYPE declaration is passed along to be=0D + #handled by the entity resolver=0D + def handle_doctype(self, name, pubid, sysid):=0D + self.ent_handler.resolveEntity(name, pubid,sysid)=0D + =0D # --- EXPERIMENTAL PYTHON SAX EXTENSIONS:=0D =0D def get_parser_name(self):=0D --==_Exmh_-3968228810-- From akuchlin@cnri.reston.va.us Sun Oct 25 03:45:13 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Sat, 24 Oct 1998 23:45:13 -0400 Subject: [XML-SIG] XML-related demo in PyGTk Message-ID: <199810250345.XAA17815@207-172-46-6.s6.tnt9.ann.erols.com> For a lark, I installed part of GNOME on my Linux machine this weekend. (GNOME is a graphical desktop environment for X that's currently in development; see www.gnome.org for information.) GNOME primarily uses the GTk widget set. James Henstridge maintains a Python interface to GTk and to the GNOME libraries, a non-trivial task since the libraries are constantly changing as they're being developed. Anyway, I downloaded the Python/GTk interface, and found that one of the demo programs displays user interfaces from the output of the GLADE UI builder ( http://www.comp.lancs.ac.uk/~damon/builder/index.html). GLADE's output is in XML, and looks like this: GtkWindow window1 destroy close_window window1 GTK_WINDOW_TOPLEVEL GTK_WIN_POS_NONE True True False ... etc ... The XML parser used is a little ad-hoc one, not xmllib.py or this SIG's package. It's an extremely interesting application, though; one could get a GUI builder for Python out of it. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ "My granny says that dying is like going to sleep," Mort added, a shade hopefully. I WOULDN'T KNOW. I HAVE DONE NEITHER. -- Mort and Death in Terry Pratchett's _Mort_ From Jeff.Johnson@stn.siemens.com Tue Oct 27 19:21:50 1998 From: Jeff.Johnson@stn.siemens.com (Jeff.Johnson@stn.siemens.com) Date: Tue, 27 Oct 1998 14:21:50 -0500 Subject: [XML-SIG] Wierd bug in DOM cloneNode (or copy.deepcopy) Message-ID: <852566AA.006A6608.00@BI01.boca.ssc.siemens.com> --0__=yExnkLfCGsa1EvhnwXSAAuTTNFW1JdjQ75rjhoLcpf6FiQ8gvGCLsUzV Content-type: text/plain; charset=us-ascii Content-Disposition: inline I have been trying to figure out what is crashing the Python interpreter on Windows NT when I ran across a strange bug. My python program runs fine on Win 98 but will crash on NT 4.0. It may take a few hours of running, but eventually it crashes. I was hoping that if I could create a small test file to reproduce the problem, I could get Guido to look into it. So... I took my DOM test file and put in an infinite loop to see if it would crash on NT. When I did, I noticed that __del__() is called more than __init__() on xml.dom.core._nodeData when cloneNode() is used. I don't know what kind of wierd voodoo happens in the copy.deepcopy() function but it looks like it messes up the reference counts somehow. That leads to a question about Python, how stable is the memory management? Should I be surprised that it runs forever on Win98 but not on WinNT? Does copy.deepcopy do things it should not? Good news! While I was writing this, my test file did crash the interpreter on NT. I'll comment out the cloneNode line and try again. Then I'm going to load NT service pack 4 and try again. The exception was in malloc(). I think I might have been out of memory. That's strange, if __del__ gets called too many times, I would think the exception would be in free(). Anyone feel like figuring this one out? (See attached file: test.py) --0__=yExnkLfCGsa1EvhnwXSAAuTTNFW1JdjQ75rjhoLcpf6FiQ8gvGCLsUzV Content-type: application/octet-stream; name="test.py" Content-Disposition: attachment; filename="test.py" Content-transfer-encoding: base64 IiIiVGhpcyB0ZXN0cyB0aGUgbmV3IFhNTCBET00gcGFja2FnZSBmb3IgY2lyY3VsYXIgcmVmZXJl bmNlIG1lbW9yeSBsZWFrcy4NClRvIHVzZSBpdCwgY29kZSBtdXN0IGJlIGFkZGVkIHRvIGNvcmUu Tm9kZSBhbmQgY29yZS5fbm9kZURhdGEgdG8gaW5jcmVtZW50DQpOb2RlX2NvdW50ZXIgaW4gX19p bml0X18gYW5kIGRlY3JlbWVudCBpdCBpbiBfX2RlbF9fLiIiIg0KDQpmcm9tIHhtbC5kb20gaW1w b3J0IGNvcmUNCmZyb20geG1sLmRvbS5zYXhfYnVpbGRlciBpbXBvcnQgU2F4QnVpbGRlcg0KZnJv bSB4bWwuc2F4IGltcG9ydCBzYXhleHRzDQpmcm9tIHhtbC5kb20ud3JpdGVyIGltcG9ydCBIdG1s V3JpdGVyDQoNCg0KZGVmIGNoZWNrTWVtKG1zZyk6DQoJaWYgY29yZS5Ob2RlLk5vZGVfY291bnRl ciBvciBjb3JlLl9ub2RlRGF0YS5Ob2RlX2NvdW50ZXI6DQoJCWlmIG1zZzogcHJpbnQgJyMjIyMj IyMjIyMjIyMjIyMjIycsbXNnLCcjIyMjIyMjIyMjIyMjIyMjIyMnDQoJCXByaW50ICJFcnJvcjog JWQgTm9kZXMgYW5kICVkIF9ub2RlRGF0YXMgd2VyZSBub3QgZGVsZXRlZCIgJSBcDQoJCQkoY29y ZS5Ob2RlLk5vZGVfY291bnRlcixjb3JlLl9ub2RlRGF0YS5Ob2RlX2NvdW50ZXIpDQoNCmRlZiB0 ZXN0MSgpOg0KCXByaW50ICc9PT09PT09PT09PT09PT09PT0gdGVzdDEgPT09PT09PT09PT09PT09 PT09Jw0KCWRvYyA9IGNvcmUuY3JlYXRlRG9jdW1lbnQoKQ0KCWRvYyA9IE5vbmUNCgljaGVja01l bSgndGVzdDEgZmFpbGVkJykNCg0KCQ0KZGVmIHRlc3QyKCk6DQoJcHJpbnQgJz09PT09PT09PT09 PT09PT09PSB0ZXN0MiA9PT09PT09PT09PT09PT09PT0nDQoJZG9jID0gY29yZS5jcmVhdGVEb2N1 bWVudCgpDQoJaHRtbCA9IGRvYy5jcmVhdGVFbGVtZW50KCdodG1sJykNCiMJYm9keSA9IGRvYy5j cmVhdGVFbGVtZW50KCdib2R5JykNCiMJaHRtbC5pbnNlcnRCZWZvcmUoYm9keSxOb25lKQ0KIwlo dG1sLmluc2VydEJlZm9yZShib2R5LE5vbmUpDQoJZG9jID0gTm9uZQ0KIwlib2R5ID0gTm9uZQ0K CWh0bWwgPSBOb25lDQoJY2hlY2tNZW0oJ3Rlc3QyIGZhaWxlZCcpDQoJDQpkZWYgdGVzdDMoKToN CglwcmludCAnPT09PT09PT09PT09PT09PT09IHRlc3QzID09PT09PT09PT09PT09PT09PScNCglk b2MgPSBjb3JlLmNyZWF0ZURvY3VtZW50KCkNCglodG1sID0gZG9jLmNyZWF0ZUVsZW1lbnQoJ2h0 bWwnKQ0KIwlkb2MuZG9jdW1lbnRFbGVtZW50ID0gaHRtbAkJIyB0aGlzIGRvZXMgbm90IGR1cGxp Y2F0ZSBodG1sIGVsZW1lbnQNCiMJZG9jLmRvY3VtZW50RWxlbWVudCA9IE5vbmUNCglkb2MgPSBO b25lDQoJaHRtbCA9IE5vbmUNCgljaGVja01lbSgndGVzdDMgZmFpbGVkJykNCg0KZGVmIHRlc3Q0 KCk6DQoJcHJpbnQgJz09PT09PT09PT09PT09PT09PSB0ZXN0NCA9PT09PT09PT09PT09PT09PT0n DQoJZG9jID0gY29yZS5jcmVhdGVEb2N1bWVudCgpDQoJaHRtbCA9IGRvYy5jcmVhdGVFbGVtZW50 KCdodG1sJykNCglodG1sLmluc2VydEJlZm9yZShkb2MuY3JlYXRlRWxlbWVudCgnYm9keScpLE5v bmUpDQoJZG9jLmFwcGVuZENoaWxkKGh0bWwpDQoJZG9jLmluc2VydEJlZm9yZShodG1sLE5vbmUp DQoJI2RvYy5kb2N1bWVudEVsZW1lbnQgPSBOb25lDQoJZG9jID0gTm9uZQ0KCWh0bWwgPSBOb25l DQoJY2hlY2tNZW0oJ3Rlc3Q0IGZhaWxlZCcpDQoNCmRlZiB0ZXN0NSgpOg0KCXByaW50ICc9PT09 PT09PT09PT09PT09PT0gdGVzdDUgPT09PT09PT09PT09PT09PT09Jw0KCWRlZiB0ZXN0NWEoKToN CgkJcCA9IHNheGV4dHMubWFrZV9wYXJzZXIoKQ0KCQlkaCA9IFNheEJ1aWxkZXIoKQ0KCQlwLnNl dERvY3VtZW50SGFuZGxlcihkaCkNCgkJcC5mZWVkKCc8SFRNTD48SEVBRC8+PC9IVE1MPicpDQoJ CWRvYyA9IGRoLmRvY3VtZW50DQoJCXAuc2V0RG9jdW1lbnRIYW5kbGVyKE5vbmUpDQoJCSNwLmNs b3NlKCkNCg0KCXRlc3Q1YSgpDQoJY2hlY2tNZW0oJ3Rlc3Q1IGZhaWxlZCcpDQoNCmRlZiB0ZXN0 NigpOg0KCSIiIlRoaXMgd2lsbCB0ZXN0IGlmIHBhcnNlci5jbG9zZSgpIGNsZWFucyB1cCB0aGUg Y2lyY3VsYXINCglyZWZlcmVuY2VzIGluIHBhcnNlci4iIiINCglwcmludCAnPT09PT09PT09PT09 PT09PT09IHRlc3Q2ID09PT09PT09PT09PT09PT09PScNCglkZWYgdGVzdDZhKCk6DQoJCXAgPSBz YXhleHRzLm1ha2VfcGFyc2VyKCkNCgkJZGggPSBTYXhCdWlsZGVyKCkNCgkJcC5zZXREb2N1bWVu dEhhbmRsZXIoZGgpDQoJCXAuZmVlZCgnPEhUTUw+PEhFQUQvPjwvSFRNTD4nKQ0KCQlkb2MgPSBk aC5kb2N1bWVudA0KCQkjcC5zZXREb2N1bWVudEhhbmRsZXIoTm9uZSkNCgkJcC5jbG9zZSgpDQoN Cgl0ZXN0NmEoKQ0KCWNoZWNrTWVtKCd0ZXN0NiBmYWlsZWQnKQ0KCQ0KZGVmIHRlc3Q3KCk6DQoJ IiIiSWYgZ2V0RWxlbWVudHNCeVRhZ05hbWUoKSBjaGVja3MgaXRzZWxmIGFzIHdlbGwgYXMgZGVz Y2VuZGFudHMNCgl0aGVuIHRoaXMgd2lsbCBwcmludCB0d28gSEVBRCBvYmplY3RzIGluc3RlYWQg b2Ygb25lLiIiIg0KCXByaW50ICc9PT09PT09PT09PT09PT09PT0gdGVzdDcgPT09PT09PT09PT09 PT09PT09Jw0KCXAgPSBzYXhleHRzLm1ha2VfcGFyc2VyKCkNCglkaCA9IFNheEJ1aWxkZXIoKQ0K CXAuc2V0RG9jdW1lbnRIYW5kbGVyKGRoKQ0KCXhtbCA9ICIiIg0KCTxIVE1MPg0KCTxIRUFEPg0K CTwvSEVBRD4NCgk8L0hUTUw+DQoJIiIiDQoJcC5mZWVkKHhtbCkNCglkb2MgPSBkaC5kb2N1bWVu dA0KCXAuc2V0RG9jdW1lbnRIYW5kbGVyKE5vbmUpDQoJI3AuY2xvc2UoKQ0KCWwgPSBkb2MuZ2V0 RWxlbWVudHNCeVRhZ05hbWUoJ0hFQUQnKQ0KCWlmIGxlbihsKSAhPSAxOg0KCQlwcmludCBsDQoJ CXByaW50ICcjIyMjIyMjIyMjIyMjIyMjIyMnLCd0ZXN0NyBmYWlsZWQnLCcjIyMjIyMjIyMjIyMj IyMjIyMnDQoNCgkNCmRlZiB0ZXN0OCgpOg0KCSIiIlRlc3RpbmcgeG1sLmRvbS5jb3JlLkVsZW1l bnQuY2xvbmVOb2RlKCkiIiINCglwcmludCAnPT09PT09PT09PT09PT09PT09IHRlc3Q4ID09PT09 PT09PT09PT09PT09PScNCgkNCglkb2MgPSBjb3JlLmNyZWF0ZURvY3VtZW50KCkNCglodG1sID0g ZG9jLmNyZWF0ZUVsZW1lbnQoJ2h0bWwnKQ0KCWJvZHkgPSBkb2MuY3JlYXRlRWxlbWVudCgnYm9k eScpDQoJdWwgPSBkb2MuY3JlYXRlRWxlbWVudCgnVUwnKQ0KCWxpID0gZG9jLmNyZWF0ZUVsZW1l bnQoJ0xJJykNCg0KDQoJZG9jLmFwcGVuZENoaWxkKGh0bWwpDQoJaHRtbC5hcHBlbmRDaGlsZChi b2R5KQ0KCWJvZHkuYXBwZW5kQ2hpbGQodWwpDQoJdWwuYXBwZW5kQ2hpbGQobGkpDQoNCgkjIFRo aXMgaXMgY2F1c2luZyBfbm9kZURhdGEgdG8gZ2V0IGZyZWVkIHR3aWNlPw0KCXVsMiA9IHVsLmNs b25lTm9kZSgxKQ0KCWJvZHkuYXBwZW5kQ2hpbGQodWwyKQ0KDQoJcHJpbnQgZG9jLnRveG1sKCkN Cg0KDQpkZWYgdGVzdDkoKToNCgkiIiJUZXN0aW5nIEh0bWxXcml0ZXIiIiINCglwcmludCAnPT09 PT09PT09PT09PT09PT09IHRlc3Q5ID09PT09PT09PT09PT09PT09PScNCgkNCglkb2MgPSBjb3Jl LmNyZWF0ZURvY3VtZW50KCkNCglodG1sID0gZG9jLmNyZWF0ZUVsZW1lbnQoJ2h0bWwnKQ0KCWJv ZHkgPSBkb2MuY3JlYXRlRWxlbWVudCgnYm9keScpDQoJdWwgPSBkb2MuY3JlYXRlRWxlbWVudCgn VUwnKQ0KCWxpID0gZG9jLmNyZWF0ZUVsZW1lbnQoJ0xJJykNCg0KCWRvYy5hcHBlbmRDaGlsZCho dG1sKQ0KCWh0bWwuYXBwZW5kQ2hpbGQoYm9keSkNCglib2R5LmFwcGVuZENoaWxkKHVsKQ0KCXVs LmFwcGVuZENoaWxkKGxpKQ0KDQoJdyA9IEh0bWxXcml0ZXIoKQ0KCXcud3JpdGUoZG9jKQ0KDQoJ cCA9IHNheGV4dHMubWFrZV9wYXJzZXIoKQ0KCWRoID0gU2F4QnVpbGRlcigpDQoJcC5zZXREb2N1 bWVudEhhbmRsZXIoZGgpDQoJeG1sID0gIiIiDQoJCTxkb2Nfc2V0IHRpdGxlPSJFV1NEIFJlbGVh c2UgMTUiIGlkPSJld3NkMTUiPg0KCQk8ZG9jX2lzc3VlIGJvb2s9IjAyMTAiIGlzc3VlPSJoMiIg ZG9jdW1lbnQ9IjAwMDEwIi8+DQoJCTxkb2NfaXNzdWUgYm9vaz0iMDIxMCIgaXNzdWU9ImwyIiBk b2N1bWVudD0iMDAwMjAiLz4NCgkJPGRvY19pc3N1ZSBib29rPSIwMjEwIiBpc3N1ZT0iZzEiIGRv Y3VtZW50PSIwMDAzMCIvPg0KCQk8L2RvY19zZXQ+DQoJCSIiIg0KCXAuZmVlZCh4bWwpDQoJZG9j ID0gZGguZG9jdW1lbnQNCglwLnNldERvY3VtZW50SGFuZGxlcihOb25lKQ0KCXcud3JpdGUoZG9j KQ0KDQoNCiNpZiBfX25hbWVfXyA9PSAnX19tYWluX18nOg0Kd2hpbGUgMToNCgl0ZXN0MSgpDQoJ dGVzdDIoKQ0KCXRlc3QzKCkNCgl0ZXN0NCgpDQoJdGVzdDUoKQ0KCXRlc3Q2KCkNCgl0ZXN0Nygp DQoJY2hlY2tNZW0oJ3Rlc3Q3IGZhaWxlZCcpDQoJdGVzdDgoKQ0KCWNoZWNrTWVtKCd0ZXN0OCBm YWlsZWQnKQ0KCXRlc3Q5KCkNCgljaGVja01lbSgndGVzdDkgZmFpbGVkJykNCg== --0__=yExnkLfCGsa1EvhnwXSAAuTTNFW1JdjQ75rjhoLcpf6FiQ8gvGCLsUzV-- From Fred L. Drake, Jr." --IGgzSjAQTa Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit I think XBEL is ready for release. I plan to send the announcement attached below to the following forums on Wednesday if there are no objections (speak quickly if there are): comp.lang.python.announce comp.text.xml xml-dev@ic.ac.uk tkbrowser-sig@python.org I've set up a SOCAT catalog at http://www.python.org/topics/xml/dtds/catalog to serve the "-//IDN python.org" prefix. I've sent a message to James Tauber (http://www.schema.net/) to add us as a DELEGATE entry to his catalog, but it doesn't appear to be there yet and I've received no reply yet. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 --IGgzSjAQTa Content-Type: text/plain Content-Description: XBEL announcement Content-Disposition: inline; filename="ANNOUNCE" Content-Transfer-Encoding: 7bit The XML Bookmark Exchange Language (XBEL) ========================================= The Python XML-SIG (http://www.python.org/sigs/xml-sig/) has developed a new XML document type for use as an Internet bookmark interchange format. The document type definition and documentation are available at: http://www.python.org/topics/xml/xbel/ XBEL supports the bookmarking capabilities of all major browsers. Demonstration conversion software is available to support Netscape Navigator, Microsoft Internet Explorer, and Opera. The Grail browser will support XBEL directly in the next release. --IGgzSjAQTa-- From Fred L. Drake, Jr." I forgot to mention: the XBEL web site is "live", just not linked to the rest of the XML topics area. Check it out at: http://www.python.org/topics/xml/xbel/ -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From jody@ldeo.columbia.edu Wed Oct 28 18:48:59 1998 From: jody@ldeo.columbia.edu (Jody Winston) Date: Wed, 28 Oct 1998 13:48:59 -0500 Subject: [XML-SIG] xml-sig and jpython Message-ID: <199810281848.NAA04781@hog> Before I start, has anyone tried to integrate the xml-sig's work with Jpython and Sun's xml-ea1 java classes? -- Jody Winston Manager SeisRes Lamont-Doherty Earth Observatory RT 9W, Palisades, NY 10964 jody@ldeo.columbia.edu From Fred L. Drake, Jr." The XML Bookmark Exchange Language (XBEL) ========================================= The Python XML-SIG (http://www.python.org/sigs/xml-sig/) has developed a new XML document type for use as an Internet bookmark interchange format. The document type definition and documentation are available at: http://www.python.org/topics/xml/xbel/ XBEL supports the bookmarking capabilities of all major browsers. Demonstration conversion software is available to support Netscape Navigator, Microsoft Internet Explorer, and Opera. The Grail browser will support XBEL directly in the next release. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From jtauber@jtauber.com Thu Oct 29 08:54:08 1998 From: jtauber@jtauber.com (James Tauber) Date: Thu, 29 Oct 1998 16:54:08 +0800 Subject: [XML-SIG] XBEL ready to go... Message-ID: <016f01be0319$d4064900$0300000a@othniel.cygnus.uwa.edu.au> -----Original Message----- From: Fred L. Drake > I've set up a SOCAT catalog at >http://www.python.org/topics/xml/dtds/catalog to serve the >"-//IDN python.org" prefix. I've sent a message to James Tauber >(http://www.schema.net/) to add us as a DELEGATE entry to his catalog, >but it doesn't appear to be there yet and I've received no reply yet. Should be up by the end of the day. James -- James Tauber / jtauber@jtauber.com / www.jtauber.com Associate Researcher, Electronic Commerce Network Curtin University of Technology, Perth, Western Australia Maintainer of : www.xmlinfo.com, www.xmlsoftware.com and www.schema.net From akuchlin@cnri.reston.va.us Thu Oct 29 15:28:15 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Thu, 29 Oct 1998 10:28:15 -0500 (EST) Subject: [XML-SIG] xml-sig and jpython In-Reply-To: <199810281848.NAA04781@hog> References: <199810281848.NAA04781@hog> Message-ID: <13880.35059.708268.20999@amarok.cnri.reston.va.us> Jody Winston writes: >Before I start, has anyone tried to integrate the xml-sig's work with Jpython and >Sun's xml-ea1 java classes? No, though I'm hoping to play with IBM's xml4j soon, with the goal of writing a Python SAX driver on top of xml4j's SAX interface. A driver for Sun's classes would also be a good thing to have. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Autumn, to me the most congenial of seasons: the University, to me the most congenial of lives. -- Robertson Davies, _The Rebel Angels_ From larsga@ifi.uio.no Thu Oct 29 17:13:39 1998 From: larsga@ifi.uio.no (Lars Marius Garshol) Date: Thu, 29 Oct 1998 18:13:39 +0100 Subject: [XML-SIG] xml-sig and jpython In-Reply-To: <13880.35059.708268.20999@amarok.cnri.reston.va.us> References: <199810281848.NAA04781@hog> <199810281848.NAA04781@hog> Message-ID: <3.0.5.32.19981029181339.00b88260@post.step.de> * Andrew M. Kuchling | | No, though I'm hoping to play with IBM's xml4j soon, with the | goal of writing a Python SAX driver on top of xml4j's SAX interface. | A driver for Sun's classes would also be a good thing to have. Uhm, I haven't looked at this at all, and won't have time for another two weeks, but why do we want to repeat this for all the 10 different Java parsers? Can't we build a general one on top of Java-SAX first? That's certainly where I would start, if possible. --Lars M. From akuchlin@cnri.reston.va.us Fri Oct 30 21:10:11 1998 From: akuchlin@cnri.reston.va.us (Andrew M. Kuchling) Date: Fri, 30 Oct 1998 16:10:11 -0500 (EST) Subject: [XML-SIG] xml-sig and jpython In-Reply-To: <3.0.5.32.19981029181339.00b88260@post.step.de> References: <199810281848.NAA04781@hog> <13880.35059.708268.20999@amarok.cnri.reston.va.us> <3.0.5.32.19981029181339.00b88260@post.step.de> Message-ID: <13882.10737.425121.545907@amarok.cnri.reston.va.us> Lars Marius Garshol writes: >Uhm, I haven't looked at this at all, and won't have time for another >two weeks, but why do we want to repeat this for all the 10 different >Java parsers? No, I meant a generic SAX driver; I didn't know whether Sun's XML parser had a SAX interface or not, and thought it might require its own specialized driver. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ Non-masochists, please delete this article NOW. -- Aaron Watters, 20 Jan 1995 From Fred L. Drake, Jr." Grail 0.5 is Released! A maintenance release of Grail is now available at the Grail web site: http://grail.cnri.reston.va.us/grail/ A few minor features have been added as well. New Features in Grail 0.5 This list is (probably) incomplete. * The bookmarks facility now supports the XBEL format for both input and output; see [1]The XBEL Bookmarks Format for more information. * The bookmarks subsystem now caches the parsed bookmarks transparently using a pickle. If the bookmarks file is changed by another application, the cache will be ignored and the bookmarks file will be re-parsed. This makes loading bookmarks much faster. Support for the explicit pickled format, available only in Grail 0.4, has been removed since no one appeared to be using it. * Bookmark metadata is now updated automatcally as you browse. In particular, the "last modified" and "last visited" fields are updated. These were not previously updated correctly. * Added support for text/paragraph and flowing text/plain MIME types. The definitions for these types are works in progress, published only as Internet Drafts. See source comments for pointers to the most recent drafts at the time of the release. * Added support for the ietf: URN scheme, also a work in progress defined by an Internet Draft. See the source for the appropriate pointer. * Added support for the doi: URN scheme, also known as the [2]Digital Object Identifier System. This system was developed by [3]CNRI for the Association of American Publishers. The work CNRI did for AAP was well received at the 1997 Frankfurt Book Fair, which led to the inception of the [4]International DOI Foundation. * Added support for [5]TkStep ([6]Linux RPM files), adjusting the widget layout for a better NeXTStep look when available. * The file: handler now reports the "last modified" data for the file. This can actually be used if you bookmarked a local file or seen in the "Document Info" window. References 1. http://grail.cnri.reston.va.us/grail/info/xbel.html 2. http://www.doi.org/ 3. http://www.cnri.reston.va.us/ 4. http://www.doi.org/welcome.html 5. http://touchwood.ee.uts.edu.au/TkSTEP/TkSTEP.html 6. http://www.fga.de/~ograf/files.shtml -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives 1895 Preston White Dr. Reston, VA 20191 From akuchlin@cnri.reston.va.us Sat Oct 31 15:08:22 1998 From: akuchlin@cnri.reston.va.us (A.M. Kuchling) Date: Sat, 31 Oct 1998 10:08:22 -0500 Subject: [XML-SIG] Wierd bug in DOM cloneNode (or copy.deepcopy) In-Reply-To: <852566AA.006A4404.00@BI01.boca.ssc.siemens.com> References: <852566AA.006A4404.00@BI01.boca.ssc.siemens.com> Message-ID: <199810311508.KAA02305@mira.erols.com> Sorry for the long time it took me to look into this; work was keeping me busy with the preparations for an important demo. Jeff.Johnson@stn.siemens.com writes: > file to reproduce the problem, I could get Guido to look into it. So... I > took my DOM test file and put in an infinite loop to see if it would crash > on NT. When I did, I noticed that __del__() is called more than __init__() > on xml.dom.core._nodeData when cloneNode() is used. I don't know what kind > of wierd voodoo happens in the copy.deepcopy() function but it looks like > it messes up the reference counts somehow. Looking through the source code of copy.deepcopy(), it will create the new _nodeData instance with the following sneaky code: y = _EmptyClass() y.__class__ = x.__class__ This will not call _nodeData's __init__ method, so the Node_counter isn't incremented, which explains the apparent discrepancy in the number of calls to __init__ and __del__; nothing is actually going wrong. Adding two __getinitargs__ methods to both classes fixes the discrepancy, but I'm not sure if it's worth checking in the change, since the Node_counter stuff is debugging code that will be taken out before the final release. > That leads to a question about Python, how stable is the memory management? > Should I be surprised that it runs forever on Win98 but not on WinNT? Does > copy.deepcopy do things it should not? It's stable, but the reference-counting garbage collection used is vulnerable to getting fooled by cycles. If you watch the program size as the infinite loop runs, you'll see that it keeps continuously growing. One problem is that some of the test cases in your code don't call p.close() as they should; I think test9() was one of them. There are also some memory leaks in xmlproc that LMG will have to look at when he gets some spare time. Using the Expat module and with p.close() being properly called, I can leave your test program running indefinitely and the amount of memory it occupies remains constant. -- A.M. Kuchling http://starship.skyport.net/crew/amk/ It is in the places where history was made that history is most sorely felt. -- Jeffrey Jacobs