[Persistence-sig] getting started

[Phillip J. Eby]

At 01:27 PM 7/10/02 -0400, Kevin Jacobs wrote:
>
>Here is a partial taxonomy of issues I'd like to see addressed.  You'll
>notice that many of them are somewhat specific to fixed schema persistent
>backends, like some object-relational (OR) systems:
>
>  1) Extensible bi-directional type mapping
>  2) Manual Schema specification vs. automatic schema introspection
>  3) Foreign key referenced object instantiation
>  4) Transactional scoping of object updates
>  5) Systems for tracking of uncommited object updates.
>  6) Query language abstraction for building OR frameworks.

These are all good points, but actually solving them is (IMHO) outside
scope for the SIG's mission.  What we want is basic support for:

1) "Transparent" persistence, for some value of "transparent".  A mechanism
to either specify that a class is intended to be persistent, or to
otherwise provide proxy or observer-style support to allow a persistence
*mechanism* to know when object states are changed or accessed, or about to
be changed or accessed, in order to do their thing.

2) Transaction framework/API, for some value of "framework/API".  Again,
this is also about mechanisms for registering, observing, or otherwise
notifying objects (or persistence mechanisms) about transaction participation.

This is actually a very narrow set of goals, ones that I think we have a
high degree of ability to achieve, if we stay focused on them, and how our
individual high-level requirements (such as you've described) are reflected
in these introspection/notification aspects.  If Python has common idioms
for dealing with these issues, then many persistence *mechanisms* can
co-exist and compete in their respective niches for what they can handle.
Perhaps multiple mechanisms might even be able to share the management of a
single object.

One reason that I believe these narrow goals are attainable, is that the
existing Zope "Persistence" and "Transaction" packages from ZODB4 *can* be
leveraged to build O-R and other mappings.  I know this, because I've
designed a framework atop the existing ZODB4 code base that can map
*anything* to or from *anything*.  Specific examples:

* Relational database
* XML/XMI in a file
* XML/XMI, persisted to a relational database
* XML/XMI, persisted in ZODB
* A relational database written using persistent objects for tables and
rows, stored in ZODB.  :)

Indeed, the design I have is of sufficient generality to persist any object
to any backend (where said backend may actually be another persistent
object, stored in yet another backend!), as long as:

1. All objects to be persisted subclass Persistence.Persistent.
2. All backends participate in the Transactions.Transaction framework.

(There is an additional restriction when dealing with backends which
themselves are stored in other backends, which is that the "outermost"
backends must support potentially commiting the same object more than once
during the tpc_begin->tpc_vote phase of transaction commit.)

(I should also note that when dealing with relational databases, my design
work addressed such matters as cache consistency, relational integrity
constraint ordering, multi-row queries, foreign key and inverse foreign key
relationships, etc., etc., ad nauseam.)

Anyway, the fact that these things can be done based solely on the existing
ZODB4 Persistent and Transaction classes, entirely ignoring the "ZODB"
package itself, means that what's available from ZODB is actually pretty
close to what's needed as a base mechanism.  It's more a question (to me,
anyway) of what could/should be improved, particularly in the form of how
the API calls and interfaces are phrased.

For my requirements, I'd be fine with it if we just put Persistent and
Transaction in the standard library, with better docs.  :)  But it'd be
nice if certain things were spelled differently, or a bit more flexible.