[issue35792] Specifying AbstractEventLoop.run_in_executor as a coroutine conflicts with implementation/intent

[Christopher Hunt]

Christopher Hunt <chrahunt at gmail.com> added the comment:

My use case is scheduling work against an executor but waiting on the results later (on demand).

If converting `BaseEventLoop.run_in_executor(executor, func, *args)` to a coroutine function, I believe there are two possible approaches (the discussion that started this [here](https://stackoverflow.com/questions/54263558/is-asyncio-run-in-executor-specified-ambiguously) only considers [impl.1]):

impl.1) `BaseEventLoop.run_in_executor` still returns a future, but we must await the coroutine object in order to get it (very breaking change), or
impl.2) `BaseEventLoop.run_in_executor` awaits on the result of `func` itself and returns the result directly

In both cases the provided `func` will only be dispatched to `executor` when the coroutine object is scheduled with the event loop.

For [impl.1], from the linked discussion, there is an example of user code required to get the behavior of schedule immediately and return future while still using `BaseEventLoop.run_in_executor`:

    async def run_now(f, *args):
        loop = asyncio.get_event_loop()
        started = asyncio.Event()
        def wrapped_f():
            loop.call_soon_threadsafe(started.set)
            return f(*args)
        fut = loop.run_in_executor(None, wrapped_f)
        await started.wait()
        return fut

however this wrapper would only be possible to use in an async function and assumes the executor is running in the same process - synchronous functions (e.g. an implementation of Protocol.data_received) would need to use an alternative `my_run_in_executor`:

    def my_run_in_executor(executor, f, *args, loop=asyncio.get_running_loop()):
        return asyncio.wrap_future(executor.submit(f, *args), loop=loop)

either of these would need to be discovered by users and live in their code base.

Having to use `my_run_in_executor` would be most unfortunate, given the purpose of `run_in_executor` per the PEP is to be a shorthand for this exact function.

For [impl.2], we are fine if the use case allows submitting and awaiting the completion of `func` in the same location, and no methods of asyncio.Future (e.g. `add_done_callback`, `cancel`) are used. If not then we still need to either:

soln.1) use `my_run_in_executor`, or
soln.2) wrap the `BaseEventLoop.run_in_executor` coroutine object/asyncio.Future with `asyncio.ensure_future`

[soln.1] is bad for the reason stated above: this is the function we are trying to avoid users having to write.

[soln.2] uses the low-level function `asyncio.ensure_future` because both of the suggested alternatives (per the docs) `asyncio.create_task` and `BaseEventLoop.create_task` throw a `TypeError` when provided an `asyncio.Future` as returned by the current implementation of `BaseEventLoop.run_in_executor`. This will have to be discovered by users and exist in their code base.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35792>
_______________________________________