[Async-sig] "read-write" synchronization

[Dima Tisnek]

Chris, here's a simple RWLock implementation and analysis:

```
import asyncio


class RWLock:
    def __init__(self):
        self.cond = asyncio.Condition()
        self.readers = 0
        self.writer = False

    async def lock(self, write=False):
        async with self.cond:
            # write requested: there cannot be readers or writers
            # read requested: there can be other readers but not writers
            while self.readers and write or self.writer:
                self.cond.wait()
            if write: self.writer = True
            else: self.readers += 1
            # self.cond.notifyAll() would be good taste
            # however no waiters can be unblocked by this state change

    async def unlock(self, write=False):
        async with self.cond:
            if write: self.writer = False
            else: self.readers -= 1
            self.cond.notifyAll()  # notify (one) could be used `if not write:`
```

Note that `.unlock` cannot validate that it's called by same coroutine
as `.lock` was.
That's because there's no concept for "current_thread" for coroutines
-- there can be many waiting on each other in the stack.

Obv., this code could be nicer:
* separate context managers for read and write cases
* .unlock can be automatic (if self.writer: unlock_for_write()) at the
cost of opening doors wide open to bugs
* policy can be introduced if `.lock` identified itself (by an
object(), since there's no thread id) in shared state
* notifyAll() makes real life use O(N^2) for N being number of
simultaneous write lock requests

Feel free to use it :)



On 26 June 2017 at 20:21, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> On Mon, Jun 26, 2017 at 10:02 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
>> Chris, coming back to your use-case.
>> Do you want to synchronise side-effect creation/deletion for the
>> sanity of side-effects only?
>> Or do you imply that callers' actions are synchronised too?
>> In other words, do your callers use those directories out of band?
>
> If I understand your question, the former. The callers aren't / need
> not be synchronized, and they aren't aware of the underlying
> synchronization happening inside the higher-level create() and
> delete() functions they would be using. (These are the two
> higher-level functions described in my pseudocode.)
>
> The synchronization is needed inside these create() and delete()
> functions since the low-level directory operations occur in different
> threads (because they are wrapped by run_in_executor()).
>
> --Chris
>
>>
>>
>> P.S./O.T. when it comes to directories, you probably want hierarchical
>> locks rather than RW.
>>
>>
>> On 26 June 2017 at 11:28, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>> On Mon, Jun 26, 2017 at 1:43 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
>>>> Perhaps you can share your use-case, both as pseudo-code and a link to
>>>> real code.
>>>>
>>>> I'm specifically interested to see why/where you'd like to use a
>>>> read-write async lock, to evaluate if this is something common or
>>>> specific, and if, perhaps, some other paradigm (like queue, worker
>>>> pool, ...) may be more useful in general case.
>>>>
>>>> I'm also curious if a full set of async sync primitives may one day
>>>> lead to async monitors. Granted, simple use of async monitor is really
>>>> a future/promise, but perhaps there are complex use cases in the
>>>> UI/react domain with its promise/stream dichotomy.
>>>
>>> Thank you, Dima. In my last email I shared pseudo-code for an approach
>>> to read-write synchronization that is independent of use case. [1]
>>>
>>> For the use case, my original purpose in mind was to synchronize many
>>> small file operations on disk like creating and removing directories
>>> that possibly share intermediate segments. The real code isn't public.
>>> But these would be operations like os.makedirs() and os.removedirs()
>>> that would be wrapped by loop.run_in_executor() to be non-blocking.
>>> The directory removal using os.removedirs() is the operation I thought
>>> should require exclusive access, so as not to interfere with directory
>>> creations in progress.
>>>
>>> Perhaps a simpler, dirtier approach would be not to synchronize at all
>>> and simply retry directory creations that fail until they succeed.
>>> That could be enough to handle rare cases where simultaneous creation
>>> and removal causes an error. You could view this an EAFP approach.
>>>
>>> Either way, I think the process of thinking through patterns for
>>> read-write synchronization is helpful for getting a better general
>>> feel and understanding of async.
>>>
>>> --Chris
>>>
>>>
>>>>
>>>> Cheers,
>>>> d.
>>>>
>>>> On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>>>> I'm relatively new to async programming in Python and am thinking
>>>>> through possibilities for doing "read-write" synchronization.
>>>>>
>>>>> I'm using asyncio, and the synchronization primitives that asyncio
>>>>> exposes are relatively simple [1]. Have options for async read-write
>>>>> synchronization already been discussed in any detail?
>>>>>
>>>>> I'm interested in designs where "readers" don't need to acquire a lock
>>>>> -- only writers. It seems like one way to deal with the main race
>>>>> condition I see that comes up would be to use loop.time(). Does that
>>>>> ring a bell, or might there be a much simpler way?
>>>>>
>>>>> Thanks,
>>>>> --Chris
>>>>>
>>>>>
>>>>> [1] https://docs.python.org/3/library/asyncio-sync.html
>>>>> _______________________________________________
>>>>> Async-sig mailing list
>>>>> Async-sig at python.org
>>>>> https://mail.python.org/mailman/listinfo/async-sig
>>>>> Code of Conduct: https://www.python.org/psf/codeofconduct/