Object oriented storage with validation

[Ilpo Nyyssönen]

"Fredrik Lundh" <fredrik at pythonware.com> writes:

> Ilpo Nyyssönen wrote:
>
>> What is the point in doing validation if it isn't done every time? Why
>> wouldn't I do it every time? It isn't that slow thing to do.
>
> DTD validation is useful in two cases:
[...]

I didn't mention DTD validation. Yes, I know the limitations of DTD
validation. DTD validation gives a clear error message with line
number in case of it doesn't match.

Show me this:

- An object oriented storage library
- A flat thing is not enough, needs some hierarchy, like in XML
- Validation that also converts the data to pythonic types, like
numbers to ints or data of my objects to my objects
- Includes a way to define version migration steps
- Parsing and validation must be fast
- Storage format preferably a readable text file
- Easy to use in application

These things would be nice to have in it too:

- Multiple backends, at least text file, XML and SQL database
- Some kind of synchronization or replication utilities

>> Pickle doesn't have validation. I am not comfortable for using it as
>> storage format that should be reliable over years when the program
>> evolves. It also doesn't tell me if my program has put something other
>> to the data than I meant to.
>
> But DTD validation doesn't tell you that either -- it's only concerned
> with the structure, not the content.
[...]

Pickle doesn't have even that. Also I can't read pickle file without
doing some program to dump it in readable format. So, I can't use
validation to make sure the data in pickle is the one I want and I
can't use less to see what is in the file. I really have NO IDEA what
is in a pickle file.

Or, yes, I clearly would need to build the validation myself on top of
it! Not going to happen.

> If you want the simplest thing, get rid of the DTD, and make your
> loader ignore things that it doesn't recognize, use default values for
> fields that are not required (or weren't in the format from the start),
> and give a nice readable error message if something required is
> missing.  That'll give you a nice, portable, reliable, and extremely
> future-proof design.

So my program will just work in the wrong way if I make a typo to a
non-required field when writing the file? No thanks.

-- 
Ilpo Nyyssönen # biny # /* :-) */