Help Create Good Data Model

[fumanchu]

I didn't say that right. As long as you are using deepcopy (or any
operation which might iterate over the keys or values in self.data),
your setter methods need that mutex, *and* it should probably be a
threading.Lock, not an RLock, just in case that iteration ends up
mutating the dict somehow. You can "get away with" no mutexes only if
*all* operations are atomic, meaning you'd have to at the least iterate
over *copies* of the dict's keys. Otherwise, you need the mutex in your
setters as well as your getters.

That turns into a performance problem quickly, which is why there's so
much literature out there on alternatives (mostly written by those
designing databases). Once you hit the performance issues, you can try
multiple dicts to simulate page locking, or cooperating locks to
implement one-writer/multiple-readers, or other techniques.

Dejavu uses a separate sandbox for each thread and lets the back end
(usually a well-written database) handle the concurrency issues.
There's a CachingProxy backend in dejavu/storage/__init__.py which has
full locks as your app does.

Of course, even atomic operations don't guarantee that overlapping
threads do what you expect. If one thread sets self.data['a'] = 1 and
another sets self.data = {}, the order of their operation may or may
not be important to you.


Robert Brewer
System Architect
Amor Ministries
fumanchu at amor.org