[issue40487] Unexpected exception handler behavior in Jupyter when returning task objects created with create_task

[John Smith]

New submission from John Smith <jeanmonet at gmail.com>:

Hi,

I'm running the following code in a Jupyter notebook (so there is already a running loop).

`run_coro` (non-async function) adds given coroutines to the loop with `loop.create_task`. I want to return the task objects created by `loop.create_task` to check for result later in the code.

Adding the task objects to a list and returning the list has the exception handler behave differently than expected.


```
async def test1():
    ''' Enumerate numbers from 1 to 9'''
    n = 0
    while (n := n + 1) < 10:
        print(f'Test 1: {n}')
        await asyncio.sleep(0.3)
    return 'Test 1: DONE'


async def test2():
    ''' Enumerate numbers starting 1, raising exception at 5'''
    n = 0
    while (n := n + 1) < 10:
        if n == 5:
            raise ValueError('Test Exception')
        print(f'Test 2: {n}')
        await asyncio.sleep(0.2)


def handle_exception(loop, context):
    if context['exception']:
        print(f'Caught exception <{context["exception"]}> while executing {context["future"]}')


def run_coro(loop, coros):
    loop.set_exception_handler(handle_exception)
    tasks = deque()
    for coro in coros:
        task = loop.create_task(coro)
        #tasks.append(task)
        # ^--- if I uncomment, exc handler behaves strangely
    return tasks
    # ^-- BUT if I remove the "return" statement, it behaves correctly again


loop = asyncio.get_running_loop()   # retrieves Jupyter's running loop

tasks = run_coro(loop, [test1(), test2()])
```

Normal (expected output):

```
Test 1: 1
Test 2: 1
Test 2: 2
Test 1: 2
Test 2: 3
Test 1: 3
Test 2: 4
Caught exception <Test Exception> while executing <Task finished name='Task-214' coro=<test2() done, defined at <ipython-input-225-b5df8e07130e>:10> exception=ValueError('Test Exception')>
Test 1: 4
Test 1: 5
Test 1: 6
Test 1: 7
Test 1: 8
Test 1: 9
```

If I uncomment `tasks.append(task)` and return the list of task objects, output is:

```
Test 1: 1
Test 2: 1
Test 2: 2
Test 1: 2
Test 2: 3
Test 1: 3
Test 2: 4
Test 1: 4
Test 1: 5
Test 1: 6
Test 1: 7
Test 1: 8
Test 1: 9
```

If I re-run the code (re-run the Jupyter cell) the exception message would appear at the beginning of the output (as if it had been waiting in a queue somewhere), so for some reason it seems to get eaten-up somewhere.

The code works normally again if I remove the return statement from the `run_coro` function.

Is this expected behaviour? Am I missing something?

----------
components: asyncio
messages: 367988
nosy: asvetlov, jeanmonet, yselivanov
priority: normal
severity: normal
status: open
title: Unexpected exception handler behavior in Jupyter when returning task objects created with create_task
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40487>
_______________________________________