问题描述
我正在学习python多处理模块，并且发现了这示例(这是一个位修改版本):
I'm learning python multiprocessing module and I've found this example (this is a bit modified version):
#!/bin/env python
import multiprocessing as mp
import random
import string
import time
# Define an output queue
output = mp.Queue()
# define a example function
def rand_string(length, output):
time.sleep(1)
""" Generates a random string of numbers, lower- and uppercase chars. """
rand_str = ''.join(random.choice(
string.ascii_lowercase
+ string.ascii_uppercase
+ string.digits)
for i in range(length))
result = (len(rand_str), rand_str)
print result
time.sleep(1)
output.put(result)
def queue_size(queue):
size = int(queue.qsize())
print size
# Setup a list of processes that we want to run
processes = [mp.Process(target=rand_string, args=(x, output)) for x in range(1,10)]
# Run processes
for p in processes:
p.start()
# Exit the completed processes
for p in processes:
p.join()
# Get process results from the output queue
results = [output.get() for p in processes]
print(results)
其输出如下:
(3, 'amF')
(1, 'c')
(6, '714CUg')
(4, '10Qg')
(5, 'Yns6h')
(7, 'wsSXj3Z')
(9, 'KRcDTtVZA')
(2, 'Qy')
(8, '50LpMzG9')
[(3, 'amF'), (1, 'c'), (6, '714CUg'), (4, '10Qg'), (5, 'Yns6h'), (9, 'KRcDTtVZA'), (2, 'Qy'), (7, 'wsSXj3Z'), (8, '50LpMzG9')]
我了解没有按创建过程的顺序(使用processes = [mp.Process(target=rand_string, args=(x, output)) for x in range(1,10)])调用过程，这在引用的文章中有所提及.我不理解(或者不确定是否理解正确)的原因是为什么result的顺序与打印输出result到STDOUT的顺序不一致?我对此的理解是，这三个操作不是原子操作(我的意思是可以通过进程开关将它们分开):
I understand that processes are not called in order which they was created (using processes = [mp.Process(target=rand_string, args=(x, output)) for x in range(1,10)]) this is mentioned in referred article. What I do not understand (or I'm not sure if understand correct) is why the order of result does not corresponds with the order in which print outputs the result to STDOUT? My understanding of this is that those three operations are not atomic (I mean that they can be separated by process switch):
print result
time.sleep(1)
output.put(result)
基本上，这里发生的是在进程print和results到STDOUT的那一刻，它切换到另一个写入results的进程.像这样的东西:
Basically what happens here is that in the moment when process print the results to STDOUT it is switched to another process which writes to results. Something like that:
Time
------------------------------------------------------------------------------------------------------------------>
Process1: print results | | | time.sleep(1) | output.put(result) |
Process2: | print results | time.sleep(1) | output.put(result) | | |
在这种情况下，STDOUT上的输出为:
In this case the output on STDOUT would be:
(1, 'c')
(2, 's5')
但是results的实际内容将是:
[ (2, 's5') (1, 'c')]
出于同样的原因，进程在创建时也不会按顺序启动.
And for the same reason the processes are not stared in order as they ware created.
我说得对吗?
推荐答案
您是对的.操作系统内核可以并且将在任何需要的时候执行上下文切换. . Python解释器(或即时编译器或其他工具)是用户空间程序，因此完全由内核控制.
You're right. The operating system kernel can and will perform context switches however and whenever it pleases to do so. The Python interpreter (or Just-In-Time compiler or whatever) is an userspace program, thus being totally controlled by the kernel.
因此，这种内核/用户奴隶制"是从父到子"传递的，换句话说，Python程序由解释器控制，而解释器由内核控制.
This "kernel/user slavery" thus is passed "from father to child", or in other words, the Python program is in merced of the interpreter, which is in turn in merced of the kernel.
这样，用户空间程序(例如Python应用程序)可以确保同步的唯一方法是使用锁定原语，例如 mutex es或其他同步原语.
As such, the only way a userspace program (such as a Python application) can assure synchronization is by the use of locking primitives, such as mutexes or other synchronization primitives.
现在，在现实世界中，通常会导致在写入文件时发生上下文切换(例如，stdout，默认情况下由print完成)，因此 lot 的操作费用较高完成，例如系统调用，复杂的内存重新映射和黑魔法，以及回送机制(例如stdout引用伪终端，这是当今最常见的情况)
Now, in real world, what usually causes a context switch in a write to a file (such as stdout, as done by print by default), a lot of expensive operations shall be done, such as system calls, complex memory remappings and black-magic-ies, and loopback mechanisms (such as when stdout refers to a pseudo-terminal, which is the most common case today).
这篇关于子流程执行的顺序及其对操作原子性的影响的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！