"RuntimeError: dictionary changed size during iteration" ; Good atomic copy operations?

[robert]

Tim Peters wrote:

> [robert]
> 
>>...
>>PS: how does ZODB work with this kind of problem? I thought is uses cPickle?
> 
> 
> It does.  Each thread in a ZODB application typically uses its own
> connection to a database.  As a result, each thread gets its own
> consistent view of database objects, which can (and routinely does)
> vary across threads.  No app-level synchronization is necessary
> because no sharing of in-memory objects occurs.  When N threads each
> load a single persistent object from its own connection, N distinct
> in-memory revisions of that object are created (one per connection ==
> one per thread).  If more than one thread modifies the same persistent
> object, the first thread to commit its changes "wins", and later
> threads that try to commit changes to the same object may suffer a
> ConflictError exception at commit time.  Between transaction
> boundaries, each thread has an independent view of database state. 
> Pragmatically, it's much more like programming with multiple processes
> than with multiple threads.

Thanks for that details.
So when committing objects with multithreaded changes on a complex 
object into ZODB, it would raise the same a RuntimeError on altered 
dicts/lists...

---

Looked up copy.py meanwhile:

copy and deepcopy use :

def _copy_dict(x):
     return x.copy()
d[types.DictionaryType] = _copy_dict

....
def _deepcopy_dict(x, memo):
     y = {}
     memo[id(x)] = y
     for key, value in x.iteritems():
         y[deepcopy(key, memo)] = deepcopy(value, memo)
     return y
d[types.DictionaryType] = _deepcopy_dict


Thus deepcopy (but not copy) seems to also expose itself to this 
RuntimeError as .iteritems() will iterate on the original dict!
( Would be maybe better to use x.items() here - as it was maybe before 
py2.2 )

Its the same Problem as with cPickle.dump. Thus there seems to be no 
RuntimeError-save possibility in the standard Python lib to get a 
"current view" of an object tree in threaded applications.

Guess it would be more wise to not expose deepcopy, cPickle.dump etc. to 
this kind of RuntimeError unnecessarily.
The speed gain of the iterator-method - if any - is minor, compared to 
the app crash problems, which are not easy to discover and work-around 
(because they happen rarely on fast computers).

Robert