Asynchronous exception handling in Python

asyncio.wait doesn’t actually consume the Futures passed to it, it just waits for them to complete, and then returns the Future objects:

coroutine asyncio.wait(futures, *, loop=None, timeout=None, return_when=ALL_COMPLETED)

Wait for the Futures and coroutine objects
given by the sequence futures to complete. Coroutines will be wrapped
in Tasks. Returns two sets of Future: (done, pending).

Until you actually yield from/await the items in the done list, they’ll remain unconsumed. Since your program exits without consuming the futures, you see the “exception was never retrieved” messages.

For your use-case, it probably makes more sense to use asyncio.gather, which will actually consume each Future, and then return a single Future that aggregates all their results (or raises the first Exception thrown by a future in the input list).

def get_runs():
    runs = [ {'name': 'one'}, {'name': 'two'} ]
    loop = asyncio.get_event_loop()
    tasks = asyncio.gather(*[fill_data(r) for r in runs])
    loop.run_until_complete(tasks)
    return runs

Output:

GET http://www.google.com/two
GET http://www.google.com/one
Exception("http://www.google.com/one has error '404: Not Found'",)

Note that asyncio.gather actually lets you customize its behavior when one of the futures raises an exception; the default behavior is to raise the first exception it hits, but it can also just return each exception object in the output list:

asyncio.gather(*coros_or_futures, loop=None, return_exceptions=False)

Return a future aggregating results from the given coroutine objects
or futures.

All futures must share the same event loop. If all the tasks are done
successfully, the returned future’s result is the list of results (in
the order of the original sequence, not necessarily the order of
results arrival). If return_exceptions is True, exceptions in the
tasks are treated the same as successful results, and gathered in the
result list; otherwise, the first raised exception will be immediately
propagated to the returned future.

Leave a Comment

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)