Server design choices

[Thomas Weholt]

Hi,

I got a Three-tier XML-based "server" ( it's not a server yet, - read on )
where a postgreSQL-database stores pieces of xml in a standardized format.
The data is fetched and updated thru a DataProvider object. In the
middel-tier the data is parsed into Python Objects and some methods are
provided to get xml-data into and out of the database, along with some
methods to parse the output of calls to the DataProvider into XML to pass on
the the application tier. In the application tier the xml is parsed into
HTML, WAP, plain text etc. and data is sent into the lower levels by using
SOAP-like ( soon real SOAP ) requests.

The problem now is this:

- each time a object is created, an instance of DataProvider is created too.
Each instance of DataProvider creates another connection to the database.
This should be done in a pooled-fashion.
- the xml for each object is stored in seperate files, not in the module
itself, and loaded on import. The reason for this is to ease updating the
xml-files. This increases time importing modules.

When all modules are loaded and connection created, the system reacts
quickly and all is well. A simple CLI test script shows this. What I don't
know is how the system will react when it is accessed thru a web-browser or
a planned CLI-based client etc. I want to create a server, a long-time
running process where all loading of modules are done and there's a pool of
postgresql-connections.

I want to use mod_python to implement the web-interface and use SOAP for all
non-browser interfaces. How is loading of modules etc done in mod_python?
when the apache -server starts or when I user calls my python-driven pages?
Would I be able to get acceptable performance at higher loads?

I also wanted to look into creating a complete SOAP-based server, where the
Apache server could connect using SOAP and format SOAP-responses into HTML
for the client. What would be the easiest way of doing this? This way the
modules would be loaded and all connections ready.

The bottom line is: how could I create a system where response time is at a
minimum? By writing a stand-alone server, independant of the apache-server
the contents going to be served by, perhaps based on SOAP or by making
simple classes that could be used in mod_python? Examples on how to create a
threaded-server for this purpose would help alot.

Any clues or hints would be great.

Thomas