[XML-SIG] XML Databases and Python

[Tony Becker]

At 15:59 +0100 1/11/02, TORZEC Nicolas thesard FTRD/DIH/LAN wrote:
>Dear all,
>I am currently involved in a project where I am using Python and where I
>manage (create /update /delete /select /use) many XML files.
>
>- My original idea was to create my own XML file managing system, but
>creating such a system will take a long time.
>- My second idea was to use a lightweight relationnal database, but it's not
>the easiest and the most efficient way to manipulate such semi-structured
>data
>- My third idea was to use a lightweight object-oriented database
>- My fourth idea was to use a lightweight native XML database

For my project, I considered or started to implement almost all of 
these ideas.  I even considered building a (4) out of a (1) or a (2). 
(3) turned out to be the most satisfactory for my case.

Roll-your-own: Building your own solution is not good because it's 
just that much more code you have to debug and maintain, on top of 
the rest of your project.

XML/Relational: Storing XML in relational databases (no matter 
exactly how this is done) seems to result in poor performance and/or 
inadequate query/update facilities, and usually ties you to a 
particular SQL platform.

Native XML: Personally, I don't think native XML databases are ready 
for prime-time yet: I'm waiting for a stable, solid definition of 
XQuery and XUpdate before I'll consider this category mature enough 
for project use.

I settled on using an object database and having my objects be 
"renderable" into XML when it's called for.  Specifically we used 
ZODB with some extra code for using Zope's "Catalogs" without the 
overhead of the rest of Zope.

This turned out to have many advantages:

- good performance if implemented properly and overall data set is small
- implement whatever kind of query you desire
- modifiers and other business logic tied more closely to data
- you can still use XSLT and other XML facilities for presentation/transmission
- the object interfaces can be exposed via XMLRPC or SOAP if you need 
these interfaces to be network-accessible
- fairly easy to write Tk/Web interfaces for editing instance data
- most of the necessary software is free, except project-specific 
things that you'd have to write yourself anyway :)

My feeling is that most XML data needs to be mutable, and there are 
always rules for how things are to be edited that cannot be enforced 
in XML editors, but require some kind of program code.  My ZODB/XML 
solution retains control over my data, while not preventing me from 
using all the XML facilities I've come to love -- my objects are 
still able to represent themselves in XML, which can be styled to FO, 
HTML, WML, etc.  (I do this by piping Python's XML output through 
Cocoon2, but it could be done in a variety of different ways.)

Best,
TB

-- 
Tony Becker, Application Architect
Fort Pedro Informatics LLC
2nd Floor East
Chicago, IL USA
http://fortpedro.com