Updating a module level shared dictionary

[Dan Stromberg]

On Tue, Jun 15, 2010 at 6:33 PM, MRAB <python at mrabarnett.plus.com> wrote:

> Vishal Rana wrote:
>
>> Hi,
>>
>> A module level dictionary 'd' and is accessed by different
>> threads/requests in a django web application. I need to update 'd' every
>> minute with a new data and the process takes about 5 seconds.
>> What could be best solution where I want the users to get either the old
>> value or the new and nothing in between.
>> I can think of a solution where a temp dictionary is constructed with a
>> new data and assigned to 'd' but now sure how this works!
>>
>> Appreciate your ideas.
>>
>>  Making changes to a structure such as a dict isn't threadsafe, but
> binding to a name is, so making a new dict, or copying the current dict and
> changing the copy, and then binding, should be OK.
>

Hmmmm...  I believe that's fine if you have one thread copying and binding,
but if one thread copies, a second thread copies, the second thread binds,
then the first thread binds - then the second thread's change gets lost.
Just because the binding is atomic doesn't mean that things related to the
binding are.

I'd probably create a dictionary-like object with a lock as an instance
attribute, and lock before and after most operations.

The issue of what to do with aggregate members is an interesting one - you
might want to avoid obtaining named references to them, actually, if that's
practical in your code.  Otherwise, you could end up putting a lock into
each aggregate (type).

treaps are supposed to be very good at dealing with parallelism - so even
though they're slower than dictionaries in uniprocessor designs, you may
find that a parallel treap would be faster. However, my treap.py module was
not written with parallelism in mind.

Actually, I think that using multithreading is probably not such a good idea
- it makes things too tightly coupled.  I prefer to use the multiprocessing
module for parallelism both because it makes much more loosely coupled
components, and because it just runs much faster than multithreading in
CPython.

On the other hand if you're using multithreading on an alternative
implementation of Python like Jython or Iron Python, the argument for using
multiprocessing isn't as strong - I'd still prefer it, but it's not quite
such a big win IMO.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100616/f7852c7e/attachment-0001.html>