Hi reliability files, writing,reading,maintaining

John Pote johnpote at blueyonder.co.uk
Thu Feb 9 07:08:43 EST 2006


Hello, help/advice appreciated.

Background:
I am writing some web scripts in python to receive small amounts of data
from remote sensors and store the data in a file. 50 to 100 bytes every 5 or
10 minutes. A new file for each day is anticipated.  Of considerable
importance is the long term availability of this data and it's gathering and
storage without gaps.

As the remote sensors have little on board storage it is important that a
web server is available to receive the data. To that end two separately
located servers will be running at all times and updating each other as new
data arrives.

I also assume each server will maintain two copies of the current data file,
only one of which would be open at any one time, and some means of
indicating if a file write has failed and which file contains valid data.
The latter is not so important as the data itself will indicate both its
completeness (missing samples) and its newness because of a time stamp with
each sample.
I would wish to secure this data gathering against crashes of the OS,
hardware failures and power outages.

So my request:
1. Are there any python modules 'out there' that might help in securely
writing such files.
2. Can anyone suggest a book or two on this kind of file management. (These
kind of problems must have been solved in the financial world many times).

Many thanks,

John Pote









More information about the Python-list mailing list