Read-Write Lock vs primitive Lock()

[Gabriel Genellina]

En Mon, 29 Dec 2008 08:17:42 -0200, k3xji <sumerc at gmail.com> escribió:

> On 29 Aralık, 11:52, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
> wrote:
>> En Mon, 29 Dec 2008 05:56:10 -0200, k3xji <sum... at gmail.com> escribió:
>>
>> > I  am trying to see on which situations does the Read-Write Lock
>> > performs better on primitive Lock() itself. Below is the code I am
>> > using to test the performance:
>> > import threading
>> > import locks
>> > import time
>>
>> > class mylock(object):
>>
>> (I'm not convinced your lock is correct)
>
> No problem.:)
>
>> > GLOBAL_VAR = 1
>> > #GLOBAL_LOCK = locks.ReadWriteLock()
>> > GLOBAL_LOCK = threading.Lock()
>> > #GLOBAL_LOCK = mylock()
>> > GLOBAL_LOOP_COUNT = 100000
>> > GLOBAL_READER_COUNT = 1000
>> > GLOBAL_WRITER_COUNT = 1
>>
>> Only one writer? If this is always the case, you don't need a lock at  
>> all.
>
> No just used for testing. It does not matter, what I want to see is
> that I want to create a botleneck on readers.

Is this a theoretical question? What do your readers/writers actually do?
Just reading a global variable does not require any locking. If it is  
being modified, you either get the previous value, or the new value.

>> Note that the thread acquires the lock ONCE, repeats several thousand
>> times an assignment to a *local* variable called GLOBAL_VAR (!), finally
>> releases the lock and exits. As every thread does the same, they just  
>> run
>> one after another, they never have a significant overlap.
>
> If I put the for loop outside, in that case, readers will not overlap
> at all, and you would be amazed by the numbers for that test. They
> indicate primitive lock is faster than read-write lock, as it requires
> the lock, executes only one bytecode operation and releases the lock.
> So, in order to create a bottlenneck on readers, we need to somehow do
> not release the lock immediately.

Again, what is your specific use case? why do you think readers will see a  
bottleneck?

> With that, primitive locks perform 10 times better than Read-Write
> lock. See above.

So what? You don't like such results? The read-write lock in the recipe  
you posted is rather complex, and isn't obvious whether it is correct or  
not, and the author doesn't prove it nor provide a reference. I'd stick to  
the standard Lock object unless I have specific needs.
Note that threading.Lock objects are implemented in C and don't have  
significant overhead (other than the wait itself); even threading.RLock  
objects are a lot slower. So depending on what you really have to do, the  
overhead of using a class like ReadWriteLock may be worse than using a  
standard Lock.

>> Hmmm, it's a reader but attempts to modify the value?
>> You don't have to protect a read operation on a global variable - so a
>> lock isn't required here.
>
> This is just for testing. Suppose that I am actually reading the
> value. I don't understand why a lock is not required? Are you saying
> lock is not necesary beacuse GLOBAL_VALUE is an immutable object, if
> then, suppose it is not. This is just a test. Suppose GLOBAl_VAR is a
> list and we are calling append() on it which is nt an atomic
> operation.

It doesn't matter whether it is mutable or immutable. Remember that, in  
Python, a global variable is just a *name* that refers to an object; you  
either get the old object referred, or the new one. If it is a list and it  
is being modified "in place", you always get the same list -- its contents  
may differ from one instant to another so iterating over it isn't safe.
That's why I insist on knowing your use case: you may me doing things more  
complicated than they should.

Attached there is a modified test using a shared list. A group of writers  
append elements; another group of writers pop elements. Readers  
continuously check the list sanity. The list starts empty, should be empty  
at the end, and should be equal to range(len(global_list)) at any time.
Run the code as-is, with all locks enabled, and it should work fine.  
Disable the locking in each place, and things start going wrong.

>> > What I am doing is: I am creating multiple readers and try to do
>> > something. I had assumed that with using primitive Lock() on the above
>> > situation, it will create a bottleneck on the rthreads. But the
>> > numbers indicate that there are no difference at all. I had
>> > implemented my own READ-WRIET lock as can be seen above mylock and
>> > also used the one here: code.activestate.com/recipes/502283/.
>>
>> I hope you now understand why you got the same numbers always.
>
> Unfortunately, I do not understand anyhing.

Each thread in your original code were like this:

begin thread; wait for lock; execute something locally; release lock; exit  
thread;

There was no shared state between threads. Even if you fix them to share  
some variable, they were a single, sequential, piece of code (the inner  
loop is irrelevant here). Once a thread gets the lock, it just runs to its  
end. No one waits more than once for the lock. So all of them run  
sequentially, in arbitrary order, but just one after the other. They never  
overlap.

-- 
Gabriel Genellina
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_locking_k3xji.py
Type: text/x-python
Size: 2344 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20081230/d8c98525/attachment-0001.py>