python-asyncio

How to use ‘yield’ inside async function?

by

Upd: Starting with Python 3.6 we have asynchronous generators and able to use yield directly inside coroutines. import asyncio async def async_generator(): for i in range(3): await asyncio.sleep(1) yield i*i async def main(): async for i in async_generator(): print(i) loop = asyncio.get_event_loop() try: loop.run_until_complete(main()) finally: loop.run_until_complete(loop.shutdown_asyncgens()) # see: https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.shutdown_asyncgens loop.close() Old answer for Python 3.5: … Read more

Why do most asyncio examples use loop.run_until_complete()?

by

run_until_complete is used to run a future until it’s finished. It will block the execution of code following it. It does, however, cause the event loop to run. Any futures that have been scheduled will run until the future passed to run_until_complete is done. Given this example: import asyncio async def do_io(): print(‘io start’) await … Read more

When using asyncio, how do you allow all running tasks to finish before shutting down the event loop

by

You can retrieve unfinished tasks and run the loop again until they finished, then close the loop or exit your program. pending = asyncio.all_tasks() loop.run_until_complete(asyncio.gather(*pending)) pending is a list of pending tasks. asyncio.gather() allows to wait on several tasks at once. If you want to ensure all the tasks are completed inside a coroutine (maybe … Read more

Asynchronous exception handling in Python

by

asyncio.wait doesn’t actually consume the Futures passed to it, it just waits for them to complete, and then returns the Future objects: coroutine asyncio.wait(futures, *, loop=None, timeout=None, return_when=ALL_COMPLETED) Wait for the Futures and coroutine objects given by the sequence futures to complete. Coroutines will be wrapped in Tasks. Returns two sets of Future: (done, pending). … Read more

Getting values from functions that run as asyncio tasks

by

The coroutines work as is. Just use the returned value from loop.run_until_complete() and call asyncio.gather() to collect multiple results: #!/usr/bin/env python3 import asyncio @asyncio.coroutine def func_normal(): print(‘A’) yield from asyncio.sleep(5) print(‘B’) return ‘saad’ @asyncio.coroutine def func_infinite(): for i in range(10): print(“–%d” % i) return ‘saad2’ loop = asyncio.get_event_loop() tasks = func_normal(), func_infinite() a, b = … Read more

Using asyncio.Queue for producer-consumer flow

by

How can I modify the program above so that the producer(s) is its own coroutine that can be scheduled concurrently with the consumers/workers? The example can be generalized without changing its essential logic: Move the insertion loop to a separate producer coroutine. Start the consumers in the background, letting them process the items as they … Read more

What kind of problems (if any) would there be combining asyncio with multiprocessing?

by

You should be able to safely combine asyncio and multiprocessing without too much trouble, though you shouldn’t be using multiprocessing directly. The cardinal sin of asyncio (and any other event-loop based asynchronous framework) is blocking the event loop. If you try to use multiprocessing directly, any time you block to wait for a child process, … Read more

What does asyncio.create_task() do?

by

What does asyncio.create_task() do? It submits the coroutine to run “in the background”, i.e. concurrently with the current task and all other tasks, switching between them at await points. It returns an awaitable handle called a “task” which you can also use to cancel the execution of the coroutine. It’s one of the central primitives … Read more