the

the

关注
发信
关注(28)粉丝(399)

如何使用python的asyncio模块正确创建和运行并发任务？

本文介绍了如何使用python的asyncio模块正确创建和运行并发任务？的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我尝试正确理解并实施两个同时运行的对象使用Python 3的相对较新的模块。



简而言之，asyncio似乎设计为处理异步进程和并发任务通过事件循环执行。它促进使用 await （在异步函数中应用）作为等待和使用结果的无回调方式，而不阻塞事件循环。 （期货和回调仍然是一个可行的替代方法。）



它还提供 asyncio.Task（）设计用于包装协程的 Future 的专用子类。最好使用 asyncio.ensure_future（）方法调用。 asyncio任务的预期用途是允许独立运行的任务与同一事件循环中的其他任务并发运行。我的理解是任务连接到事件循环，然后自动保持在 await 语句之间的协程。 p>

我喜欢能够使用并发任务而不需要使用类，但我没有发现很多实现的细节。



这是我目前的做法：

  import asyncio 
 
 print（'running async test'）
 
 async def say_boo（）：
i = 0 
 while True：
 await asyncio.sleep（0）
 print（'... boo {0}'。format（i））
i + = 1 
 
 async def say_baa（）：
i = 0 
 while True：
 await asyncio.sleep（0）
 print（'... baa {0}'。format（i）） 
i + = 1 
 
＃选项1：换行任务对象
＃ - >自动附加到事件循环并执行
 boo = asyncio.ensure_future（say_boo（））
 baa = asyncio.ensure_future（say_baa（））
 
 loop = asyncio.get_event_loop 
 loop.run_forever（）



在尝试同时运行两个循环任务的情况下，我注意到，除非任务有一个内部的 await 表达式，它会陷入 while 循环，有效阻止其他任务运行（很像一个正常的 while 循环）。然而，一旦任务需要（a）等待，它们似乎同时运行没有问题。



因此， await 语句似乎为事件循环提供了在任务之间来回切换的立足点



内部 await 的示例输出：

 运行异步测试
 ... boo 0 
 ... baa 0 
 ... boo 1 
。 ..baa 1 
 ... boo 2 
 ... baa 2

示例输出没有内部 await ：

  ... boo 0 
 ... boo 1 
 ... boo 2 
 ... boo 3 
 ... boo 4



问题



此实现是否适用于并发循环 asyncio 中的任务？



这是正确的，唯一的方法是为任务提供一个阻塞点$ c> await expression），以便事件循环处理多个任务？

解决方案

，在事件循环中运行的任何协程将阻止其他协程和任务运行


  1. 使用<$ c $调用另一个协同程序

  2. 返回
    3. >

这是因为 asyncio 是单线程的;事件循环运行的唯一方法是没有其他协同程序正在执行。使用 yield from / await 暂时挂起协程，使事件循环有机会工作。



您的示例代码很好，但在许多情况下，您可能不想要长时间运行的代码，而不是在事件循环中运行异步I / O。在这些情况下，使用在后台线程或进程中运行代码。 ProcessPoolExecutor 将是更好的选择，如果你的任务是CPU限制， ThreadPoolExecutor 将被使用，如果你需要做一些I / O不是 asyncio 友好。



例如，你的两个循环完全是CPU-并且不共享任何状态，因此最佳性能将来自使用 ProcessPoolExecutor 在CPU之间并行运行每个循环：

  import asyncio 
从concurrent.futures导入ProcessPoolExecutor 
 
 print（'running async test'）
 
 def say_boo（）：
i = 0 
 while True：
 print（'... boo {0}'。format（i））
i + = 1 
 
 
 def say_baa（）：
i = 0 
 while True：
 print（'... baa {0}'。format（i））
i + = 1 
 
如果__name__ ==__main__：
 executor = ProcessPoolExecutor（2）
 loop = asyncio.get_event_loop（）
 boo = asyncio.ensure_future （loop.run_in_executor（executor，say_boo））
 baa = asyncio.ensure_future（loop.run_in_executor（executor，say_baa））
 
 loop.run_forever（）


I am trying to properly understand and implement two concurrently running Task objects using Python 3's relatively new asyncio module.

In a nutshell, asyncio seems designed to handle asynchronous processes and concurrent Task execution over an event loop. It promotes the use of await (applied in async functions) as a callback-free way to wait for and use a result, without blocking the event loop. (Futures and callbacks are still a viable alternative.)

It also provides the asyncio.Task() class, a specialized subclass of Future designed to wrap coroutines. Preferably invoked by using the asyncio.ensure_future() method. The intended use of asyncio tasks is to allow independently running tasks to run 'concurrently' with other tasks within the same event loop. My understanding is that Tasks are connected to the event loop which then automatically keeps driving the coroutine between await statements.

I like the idea of being able to use concurrent Tasks without needing to use one of the Executor classes, but I haven't found much elaboration on implementation.

This is how I'm currently doing it:

import asyncio

print('running async test')

async def say_boo():
    i = 0
    while True:
        await asyncio.sleep(0)
        print('...boo {0}'.format(i))
        i += 1

async def say_baa():
    i = 0
    while True:
        await asyncio.sleep(0)
        print('...baa {0}'.format(i))
        i += 1

# OPTION 1: wrap in Task object
# -> automatically attaches to event loop and executes
boo = asyncio.ensure_future(say_boo())
baa = asyncio.ensure_future(say_baa())

loop = asyncio.get_event_loop()
loop.run_forever()

In the case of trying to concurrently run two looping Tasks, I've noticed that unless the Task has an internal await expression, it will get stuck in the while loop, effectively blocking other tasks from running (much like a normal while loop). However, as soon the Tasks have to (a)wait, they seem to run concurrently without an issue.

Thus, the await statements seem to provide the event loop with a foothold for switching back and forth between the tasks, giving the effect of concurrency.

Example output with internal await:

running async test
...boo 0
...baa 0
...boo 1
...baa 1
...boo 2
...baa 2

Example output without internal await:

...boo 0
...boo 1
...boo 2
...boo 3
...boo 4

Questions

Does this implementation pass for a 'proper' example of concurrent looping Tasks in asyncio?

Is it correct that the only way this works is for a Task to provide a blocking point (await expression) in order for the event loop to juggle multiple tasks?

解决方案

Yes, any coroutine that's running inside your event loop will block other coroutines and tasks from running, unless it

  1. Calls another coroutine using yield from or await (if using Python 3.5+).
  2. Returns.

This is because asyncio is single-threaded; the only way for the event loop to run is for no other coroutine to be actively executing. Using yield from/await suspends the coroutine temporarily, giving the event loop a chance to work.

Your example code is fine, but in many cases, you probably wouldn't want long-running code that isn't doing asynchronous I/O running inside the event loop to begin with. In those cases, it often makes more sense to use BaseEventLoop.run_in_executor to run the code in a background thread or process. ProcessPoolExecutor would be the better choice if your task is CPU-bound, ThreadPoolExecutor would be used if you need to do some I/O that isn't asyncio-friendly.

Your two loops, for example, are completely CPU-bound and don't share any state, so the best performance would come from using ProcessPoolExecutor to run each loop in parallel across CPUs:

import asyncio
from concurrent.futures import ProcessPoolExecutor

print('running async test')

def say_boo():
    i = 0
    while True:
        print('...boo {0}'.format(i))
        i += 1


def say_baa():
    i = 0
    while True:
        print('...baa {0}'.format(i))
        i += 1

if __name__ == "__main__":
    executor = ProcessPoolExecutor(2)
    loop = asyncio.get_event_loop()
    boo = asyncio.ensure_future(loop.run_in_executor(executor, say_boo))
    baa = asyncio.ensure_future(loop.run_in_executor(executor, say_baa))

    loop.run_forever()

这篇关于如何使用python的asyncio模块正确创建和运行并发任务？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

06-29 05:37
Powered by LMLPHP ©2024 the 0.042885