[Async-sig] "read-write" synchronization

[Nathaniel Smith]

On Mon, Jun 26, 2017 at 12:37 PM, Dima Tisnek <dimaqq at gmail.com> wrote:
> Note that `.unlock` cannot validate that it's called by same coroutine
> as `.lock` was.
> That's because there's no concept for "current_thread" for coroutines
> -- there can be many waiting on each other in the stack.

This is also a surprisingly complex design question. Your async RWLock
actually matches how Python's threading.Lock works: you're explicitly
allowed to acquire it in one thread and then release it from another.
People sometimes find this surprising, and it prevents some kinds of
error-checking. For example, this code *probably* deadlocks:

    lock = threading.Lock()
    lock.acquire()
    # probably deadlocks
    lock.acquire()

but the interpreter can't detect this and raise an error, because in
theory some other thread might come along and call lock.release(). On
the other hand, it is sometimes useful to be able to acquire a lock in
one thread and then "hand it off" to e.g. a child thread. (Reentrant
locks, OTOH, do have an implicit concept of ownership -- they kind of
have to, if you think about it -- so even if you don't need reentrancy
they can be useful because they'll raise a noisy error if you
accidentally try to release a lock from the wrong thread.)

In trio we do have a current_task() concept, and the basic trio.Lock
[1] does track ownership, and I even have a Semaphore-equivalent that
tracks ownership as well [2]. The motivation here is that I want to
provide nice debugging tools to detect things like deadlocks, which is
only possible when your primitives have some kind of ownership
tracking. So far this just means that we detect and error on these
kinds of simple cases:

    lock = trio.Lock()
    await lock.acquire()
    # raises an error
   await lock.acquire()

But I have ambitions to do more [3] :-).

However, this raises some tricky design questions around how and
whether to support the "passing ownership" cases. Of course you can
always fall back on something like a raw Semaphore, but it turns out
that trio.run_in_worker_thread (our equivalent of asyncio's
run_in_executor) actually wants to do something like pass ownership
from the calling task into the spawned thread. So far I've handled
this by adding acquire_on_behalf_of/release_on_behalf_of methods to
the primitive that run_in_worker_thread uses, but this isn't really
fully baked yet.

-n

[1] https://trio.readthedocs.io/en/latest/reference-core.html#trio.Lock
[2] https://trio.readthedocs.io/en/latest/reference-core.html#trio.CapacityLimiter
[3] https://github.com/python-trio/trio/issues/182

-- 
Nathaniel J. Smith -- https://vorpus.org