Python Tornado - 对如何将阻塞函数转换为非阻塞函数感到困惑

问题描述

假设我有一个长时间运行的函数:

Suppose I have a long running function:

def long_running_function():
    result_future = Future()
    result = 0
    for i in xrange(500000):
        result += i
    result_future.set_result(result)
    return result_future

我在处理程序中有一个 get 函数，该函数使用 for 循环的上述结果打印用户，该循环添加了 xrange 中的所有数字:

I have a get function in a handler that prints the user with the above result of a for loop that adds all the number in the xrange:

@gen.coroutine
def get(self):
    print "start"

    self.future = long_running_function()
    message = yield self.future
    self.write(str(message))

    print "end"

如果我同时在两个网络浏览器上运行上面的代码，我得到:

If I run the above code on two web browsers simultaneously, I get:

开始

结束

开始

结束

这似乎是阻塞的.根据我的理解，@gen.coroutine 和 yield 语句不会阻塞 get 函数中的 IOLoop，但是，如果协程中的任何函数正在阻塞，然后它阻塞了 IOLoop.

Which seems to be blocking. From my understanding, the @gen.coroutine and the yield statement does not block the IOLoop in the get function, however, if any functions that is inside the co-routine that is blocking, then it blocks the IOLoop.

因此我做的另一件事是将 long_running_function 转换为回调，并使用 yield gen.Task 代替.

Hence the other thing I did is to turn the long_running_function into a callback, and using the yield gen.Task instead.

@gen.coroutine
def get(self):
    print "start"

    self.future = self.long_running_function
    message = yield gen.Task(self.future, None)
    self.write(str(message))

    print "end"

def long_running_function(self, arguments, callback):
    result = 0
    for i in xrange(50000000):
        result += i
    return callback(result)

这也没有削减，它给了我:

This doesn't cut too, it gives me:

开始

结束

开始

结束

我可以使用线程来并行执行它们，但这似乎不是可行的方法，因为我可能要打开很多线程，而且根据 Tornado 的用户指南，这可能很昂贵.

I can use threads to execute those in parallel, but it doesn't seem the way to go, because I might be opening a lot of threads, and according to Tornado's user guide, it may be expensive.

人们如何为 Tornado 编写异步库?

How do people write async libraries for Tornado?

推荐答案

如果阻塞函数受 CPU 限制(如您的 for/xrange 示例)，那么线程(或进程)是使其非阻塞.为每个传入请求创建一个线程的开销很大，但创建一个小的 ThreadPoolExecutor 来处理所有 CPU 密集型操作的开销不大.

If the blocking function is CPU-bound (as your for/xrange example is), then threads (or processes) are the only way to make it non-blocking. Creating a thread per incoming request is expensive, but making a small ThreadPoolExecutor to handle all CPU-bound operations is not.

要在不使用线程的情况下使函数非阻塞，该函数必须事件驱动:它必须等待某些外部事件(例如网络 I/O)，以便它可以当该事件发生时被唤醒.

To make a function non-blocking without using threads, the function must be event-driven: it must be waiting on some external event (such as network I/O) so that it can be awoken when that event occurs.

这篇关于Python Tornado - 对如何将阻塞函数转换为非阻塞函数感到困惑的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！