IndexedCatalog and ZEO

[Diez B. Roggisch]

> I'm trying to use IndexedCatalog
> [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
> [http://www.cherrypy.org] application.
> As Zodb support access just from one Python thread, I must either use just
> one CherryPy thread (slow), or use ZEO. However, if I understand it good,
> when I use
> from ZEO import ClientStorage
> 
> then ZEO is internally using FileStorage, so no IndexedCatalog is used.
> 
> Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
> ZEO for myself (or run this one thread)?

I'm not sure that I fully understand your problem - the usage of
IndexedCatalog is IMHO unrelated to threading issues of zodb. And it's not
true that zodb supports only one thread. I use it in a multi-threaded
environment - as does zope. 

But it is necessary to use a separate connection object per thread. I've
written myself a transaction service alike to the java transaction api that
creates a connection for every call made to my server. In conjunction with
proxy-objects that keep a unique key for each object, I can happily access
zodb concurrently.

The basic idea is like this:

# open a zodb file storage and store a global reference
db = ....

def get_root():
    # the module threadlocal is from the cookbook
    conn = threadlocal.get("connection", None)
    if conn is None:
        conn = db.open()
        threadlocal["connection"] = conn
    return conn.root()

class Data(PersistenObject):
    """ my data-object. It has a property id that identifies it uniquely """
    ...
 
class DataProxy(object):
    def __init__(self, id):
         self.id = id

    def _g_data(self):
        return get_root()[self.id]

    data = property(_g_data)

    def __getattr__(self, name):
        return getattr(self.data, name)

The magic is in the get_root()-function that fetches a thread-local
connection and returns the root-object in my zodb. In my real application,
the data-object is cached as long as I'm in one transaction, so multiple
attribute accesses are faster.

I have to add that I have no expirience with zeo - so maybe that can also
solve your problems, but that will result in a local loop network access
that also slows down your application.

Regarding the usage of IndexedCatalog: It seems you can use it in
conjunction with zeo as it has no idea of whatever storage is needed - all
it does is indexing zodb objects - which you get from zeo as well. Of
course that means that you have to keep a catalog for every thread/process
that accesses the objects. Alternatively, you maybe can make the
IndexedCatalog a  zodb-stored object itself, but I'm not sure about that.

-- 
Regards,

Diez B. Roggisch