Hi reliability files, writing,reading,maintaining

[Cameron Laird]

In article <43eb4480$0$11064$e4fe514c at news.xs4all.nl>,
Martin P. Hellwig <mhellwig at xs4all.nl> wrote:
>John Pote wrote:
><cut>
>> So my request:
>> 1. Are there any python modules 'out there' that might help in securely
>> writing such files.
>> 2. Can anyone suggest a book or two on this kind of file management. (These
>> kind of problems must have been solved in the financial world many times).
>> 
><cut>
>I can't answer your specific questions but I got the feeling that you're 
>  barking at the wrong tree ;-)
>
>You don't want to solve this in you application, file management is what 
>the OS and hardware is about. "Military" grade solutions are often 
>(depending on their criticalness) double or triple hot spares in the 
>same room which can be seen as a "unit" and these units are duplicated 
>on remote locations (at least 25km of each other) syncing their data 
>with standard tools like rsync.
>
>If you don't have military budget, I would suggest to do it a little 
>less expensive, like having a couple of Solaris machines (three or four 
>will do it) in the same room, using a part of their diskspace for ZFS.
>
>Then let your application write your data to that ZFS partition and if 
>you are particular paranoid you can build in a checksum that can be 
>calculated by other machines without the need for the original received 
>data (ZFS has a built-in mechanism for that so you might just want to 
>call that).
>
>There is nothing wrong for assuming a certain level of hardware, well at 
>least not if its very clearly communicated to all parties.
>ZFS is open source (by SUN) but currently only implemented in Solaris, 
>which you also can (commercially) use without charge.
>
>Now you only got to figure out how to implement a heartbeat mechanism 
>between you fail-over applications :-)
			.
			.
			.
Also, the whole idea of basing reliability on HTTP uploads of 50 bytes
at a time sounds to me a bit ... strained.  There *must* be simpler
ways--and simpler goes a long way in the direction of trustworthy.