Object Database (ODBMS) for Python

[Paul D. Fernhout]

Patrick K. O'Brien wrote:
> Let me start by saying I'd love to cooperate, even if I am
> competitive by nature.  ;-)

Nothing like a good controversy to get people paying attention. :-)

> This API looks rather verbose to me.  I think mine would look like:
>>>> t = tx.Create('User', name='Sir Galahad') user = db.execute(t)

I think your notion of transactions is growing on me. :-) I can see how
you can generalize this to construct a transaction in a view of a
database, querying on DB + T1 + T2 etc. while they are uncommitted and
then commit them all (perhaps resolving multiuser multitransaction
issues on commits). Kind of neat concept, I'll have to consider for some
version of the Pointrel System.

I think it is the special syntax of:
  tx.Update(u1, name='Joe')
or:
  tx.Create('User', name='Sir Galahad')
which I am recoiling some from.

I think part of this comes from thinking as a transaction as something
that encloses other changes, as opposed to something which is changed.
Thus my discomfort at requesting services from a transaction other than
commit or abandon. I'm not saying maybe I couldn't grow to love
tx.Update(), just that it seems awkward at first compared to what I am
used to, as well compared to making operations on a database itself
after having told the database to begin a transaction. I'm also left
wondering what the read value of the "name" field is when accessed
directly as "u1.name" after doing the "wx.Update()" and before doing the
"db.execute()". [By the way, pickly, picky, and I fall down on it too,
but you use different capitalizations for those two functions.]

So is it that in PyPerSyst there appears to be one way to access
information (directly through the object using Python object attribute
access dot syntax) [not sure about database queries?] and another way to
change objects -- using tx.XYZ()? This mixing of mindsets could be
confusing (especially within an object that changes its own values
internally).

Using tx.Update also becomes an issue of how to convert existing code to
persistant code.  Mind you, the Pointrel System can't do this
transparently either, but it doesn't try to do it at all. The Pointrel
System requires both looking up a value and storing it to use a
different syntax. Is it just a matter of aesthetics about whether it is
better to have the whole approach be unfamiliar or whether it is better
to have only half of it be unfamiliar? Or is there something more here,
some violation of programmer expectations? [See below.]

> And unique ids (immutable, btw) are assigned by PyPerSyst:
>>>> user.oid
> 42

Being competetive here :-) I would love to know if you have a good
approach for making them globally unique across all possible users of
all PyPerSyst repositories for all time. The Pointrel has an approach to
handle this (I don't say it will always work, or is efficient, but it
tries). :-) Feel free to raid that code (BSDish license, see
license.txt), but that issue may have other deeper implications for your
system.

> And you can still access attributes directly, you just can't change 
> them outside of a transaction:
> 
> 
>>>> user.name
> 
> 'Sir Galahad'
> 
> And the generic Update transaction is equally simple:
> 
> 
>>>> t = tx.Update(user, name='Brian') db.execute(t) user.name
> 'Brian'

I know one rule of user interface design (not nexceesarily API of
course) is that familiar elements should act familiar (i.e. a drop down
list should not launch a dialog window on drop down) and that if you are
going to experiment it should look very different so expectations are
not violated.

The issue here is in part that when you can reference "u1.name" and then
"u1.name = 'Joe'" generates an exception (instead of automatically
making an implict transaction), some user expectation of API symmetry
may be violated...

Also, on another issue, it seems like the persistant classes need to
derive from a special class and define their persistant features in a
special wy, i.e. class Realm(Entity): _attrSpec = [ 'name', ] etc.
Again, this is going somewhat towards Python language integration yet
not all the way.

While I'd certainly agree your version is more concise than what I
posted first (just an example of a system that does not attempt to use
Python language features), later in the email (perhaps you'll get to it
in your next reply) was the simpler:

   import persistanceSystem import *
   foo = MyClass()
   PersistanceSystem_Wrap(foo)
   # the following defaults to a transaction
   foo.x = 10
   # this makes a two change transaction
   PersistanceSystem_StartTransaction()
   foo.y = 20
   foo.z = 20
   foo.info = "I am a 3D Point"
   PersistanceSystem_EndTransaction()

That approach does not violate any symmetry expectations by users -- you
can assign and retrieve values just like always.

>> Granted, the Pointrel System is essentially a single user single 
>> transaction system at the core. It (in theory, subject to bugs) 
>> supports atomicity (transactions), isolation (locking) and 
>> durability (logging&recovery). It only supports consistency by how 
>> applications use transactions as opposed to explicit constraints or
>>  rules maintained by the database, so one could argue it fails the 
>> ACID test there. (Although would any typical ODBMS pass consistency
>>  without extra code support? Does PyPerSyst have this as the
>> database level?)
> 
> 
> PyPerSyst can persist *any* picklable object graph.

Are the graphs stand alone can they reference other previously persisted
Python objects (not derived from "Root" or "Entity")?

> But it also comes with an Entity class and a Root class (that
> understands Entity classes) that provides additional functionality,
> such as alternate indexes, referential integrity, instance
> validation, etc.

I guess I need to learn more about when these are better handled by the
persistance system as opposed to the applications that use it.

> I don't mind a friendly challenge.  I'm just surprised that the bulk 
> of this thread is debating an API that has barely seen the light of 
> day, and that I consider to be drop-dead simple.  I guess I need to 
> get a demo app created soon, just to put this to rest.  Or at least 
> make sure we're all debating about the same thing.  ;-)

Good point.

I think the issue is that with the other systems out there
(MySQL, ZODB, etc.) it seems like a new system has to offer something
really new (speed, footprint, simplicity, robustness, documentation :-) 
etc.).

Presumably a very transaparent API for persistance is still needed for 
an ODBMS which is Python friendly? (Does ZODB do any of this?) If I need 
to write any extra code at all for an object to be persistant, or derive 
from a specialized class, I could just derive from a class that knows 
how to use SQL to store pickled fields. Obviously, PyPerSyst may have 
many wonderful features (not having used it yet) which make it worth it 
to do a special derivation or write special code, but it just seems like 
it would have language transparency too. But, I haven't tried to do that 
in Python, so maybe it's not possible.

> Right now we're debating an API that nobody on this thread has really
>  seen or used, other than me.  The other thing I can say is that,
> imo, the way you interact with persistent class instances is not the
> same way you interact with regular class instances.  Not if you value
> the integrity and reliability of your data.  And trying to make it
> appear so is a disservice.  I know everyone seems to think
> transparent persistence is the holy grail, but I've come to think
> otherwise.

I think this is the core of the question of this part of the thread.
You wrote "I've come to think otherwise". I'd be curious to hear more on
any use cases or examples on why transaparency is not so compatible with
reliability etc. I frankly don't know. I just don't see them being
mutually exclusive, especially based on what I have read of Smalltalk
systems that do persistance using proxies. But again, Smalltalk has
"become:" which can essentially swap any arbitray instance and a proxy,
thus making it easy to suddenly start using a proxy for a previously
used instance and have all previous references point to the proxy. Maybe 
Python need's become? I could use it elsewhere. Maybe it has it and I 
never noticed?

> Unfortunately, I don't have time to fully elaborate my position.  But
>  you don't have to agree with me on this point.  PyPerSyst is very 
> modular, and there implementations of transparent proxies in the 
> PyPerSyst CVS sandbox that some other developers on the team have 
> written.  So it can be done.

OK.

Thanks for the reply.

--Paul Fernhout
http://www.pointrel.org



-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 100,000 Newsgroups - 19 Different Servers! =-----