Object oriented storage with validation (was: Re: Caching compiled regexps across sessions (was Re: Regular Expressions - Python vs Perl))

[Fredrik Lundh]

Ilpo Nyyssönen wrote:

> What is the point in doing validation if it isn't done every time? Why
> wouldn't I do it every time? It isn't that slow thing to do.

DTD validation is useful in two cases: making sure that data from
a foreign source has the right structure, and making sure that data
you create has the right structure.  The former is relevant for de-
ployed code, but the latter really only makes sense during deve-
lopment, and can easily be solved by running an external validator
as part of your test suite.

> Pickle doesn't have validation. I am not comfortable for using it as
> storage format that should be reliable over years when the program
> evolves. It also doesn't tell me if my program has put something other
> to the data than I meant to.

But DTD validation doesn't tell you that either -- it's only concerned
with the structure, not the content. You can get a bit further with better
schema technologies, but if you want reliable storage, use checksums
or digests.  Validation is like the helmet used by skydivers; if you think
that's all you need, you sure is going to be surprised when you hit the
ground.

> I want to do the simplest thing, but I also want something that helps
> me keep the program usable also in the future. I prefer putting some
> resources to get some validation to it initially than use later more
> resouces to do something with undetermined lump of data.

If you want the simplest thing, get rid of the DTD, and make your
loader ignore things that it doesn't recognize, use default values for
fields that are not required (or weren't in the format from the start),
and give a nice readable error message if something required is
missing.  That'll give you a nice, portable, reliable, and extremely
future-proof design.

> > Check out (coincidentally) Fredrik's elementtree:
> >
> > http://effbot.org/zone/element-index.htm
>
> At least the interface looks quite simple and usable. With some
> validation wrapping over it, it might be ok...

I was going to point you to a validating parser for ET, but the "it might
be ok" statement is a bit too arrogant for my taste.

</F>