How to handle async and inheritance?

[Stephen Rosen]

Hi all,

I'm looking at a conflict between code sharing via inheritance and async
usage. I would greatly appreciate any guidance, ideas, or best practices
which might help.

I'll speak here in terms of a toy example, but, if anyone wants to look at
the real code, I'm working on webargs. [1] Specifically, we have a `Parser`
class and an `AsyncParser` subclass, and the two have a lot of code
duplication to handle async/await. [2]


I've got an inheritance structure like this:

class MyAbstractType: ...
class ConcreteType(MyAbstractType): ...
class AsyncConcreteType(MyAbstractType): ...

One of my goals, of course, is to share code between ConcreteType and
AsyncConcreteType via their parent.
But the trouble is that there are functions defined like this:

class MyAbstractType:
    def foo(self):
        x = self.bar()
        y = self.baz(x)
        ... # some code here, let's say 20 lines

class AsyncConcreteType(MyAbstractType):
    async def foo(self):
        x = await self.bar()
        y = self.baz(x)
        ... # the same 20 lines as above, but with an `await` added
every-other line


I'm aware that I'm looking at "function color" and that my scenario is
pitting two language features -- inheritance and async -- against one
another. But I don't see a clean way out if we want to support an
"async-aware" version of a class with synchronous methods.

What I tried already, which I couldn't get to work, was to either fiddle
with things like `inspect` to see if the current function is async or to
use a class variable to indicate that the current class is the async
version. The idea was to write something like

class MyAbstractType:
    _use_async_calls = False
    def foo(self):
        x = self._await_if_i_am_async(self.bar)
        y = self.baz(x)
        ...

and that way, the async subclass just needs to change signatures to be
async with little stubs and set the flag,

class AsyncConcreteType(MyAbstractType):
    _use_async_calls = True
    async def foo(self):
        return super().foo()
    async def bar(self):
        return super().bar()

but this (some of you are ahead of me on this, I'm sure!) did not work at
all. I couldn't find any way to write `_await_if_i_am_async`, other than
possibly doing some weird things with `exec`.
Once I concluded that the python wouldn't let me decide whether or not to
use await at runtime, at least with tools of which I'm aware, I basically
gave up on that route.


However, it seems like there should be some clever technique for defining
`MyAbstractType.foo` such that it awaits on certain calls *if* there's some
indication that it should do so. It's obviously possible with `exec`, but I
don't want to convert all of the core codepaths into giant `exec` blocks.
Perhaps there's a way which is safer and more maintainable though?


If anyone has experience in this space and can offer up a good solution, I
would love to hear about it.
And if someone wants to go above-and-beyond and look at webargs, and
suggest a better way for us to support aiohttp, I'd obviously welcome that
kind of help as well!

Thanks in advance, and best regards,
-Stephen


[1] https://github.com/marshmallow-code/webargs
[2]
https://github.com/marshmallow-code/webargs/blob/6668d267fa4135cf3f653e422bd168298f2213a8/src/webargs/asyncparser.py#L24