GNU Parallel和Bash函数:如何从手册中运行简单示例

问题描述

我正在尝试学习GNU Parallel，因为我认为我可以轻松地并行化bash函数.因此，在尝试学习时，我去了 GNU并行手册，其中有一个示例 ...但是我什至无法使它正常工作！发挥作用:

I'm trying to learn GNU Parallel because I have a case where I think I could easily parallelize a bash function. So in trying to learn, I went to the GNU Parallel manual where there is an example...but I can't even get it working! To wit:

(232) $ bash --version
GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
(233) $ cat tpar.bash
#!/bin/bash

echo `which parallel`
doit() {
  echo Doing it for $1
  sleep 2
  echo Done with $1
}
export -f doit
parallel doit ::: 1 2 3
doubleit() {
  echo Doing it for $1 $2
  sleep 2
  echo Done with $1 $2
}
export -f doubleit
parallel doubleit ::: 1 2 3 ::: a b

(234) $ bash tpar.bash
/home/mathomp4/bin/parallel
doit: Command not found.
doit: Command not found.
doit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.

如您所见，我什至无法获得运行该示例的简单方法.因此，我可能正在做一些非常愚蠢而基本的事情……但是我很茫然.

As you can see, I can't even get the simple example to run. Thus, I'm probably doing something amazingly stupid and basic...but I'm at a loss.

ETA:如评论员所建议(chmod + x，设置-vx):

ETA: As suggested by commenters (chmod +x, set -vx):

(27) $ ./tpar.bash

echo `which parallel`
which parallel
++ which parallel
+ echo /home/mathomp4/bin/parallel
/home/mathomp4/bin/parallel

doit() {
  echo Doing it for $1
  sleep 2
  echo Done with $1
}
export -f doit
+ export -f doit
parallel doit ::: 1 2 3
+ parallel doit ::: 1 2 3
doit: Command not found.
doit: Command not found.
doit: Command not found.
doubleit() {
  echo Doing it for $1 $2
  sleep 2
  echo Done with $1 $2
}
export -f doubleit
+ export -f doubleit
parallel doubleit ::: 1 2 3 ::: a b
+ parallel doubleit ::: 1 2 3 ::: a b
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.
doubleit: Command not found.

ETA2:注意，我可以在脚本中只说'doit 1'，它将做到这一点.因此该功能有效，只是...不被导出?

ETA2: Note, I can, in the script, just call 'doit 1', say, and it will do that. So the function is valid, it just isn't...exported?

推荐答案

您不能从定义了shell的外部调用shell函数.壳函数是壳内部的一个概念. parallel命令本身无法访问它.

You cannot call a shell function from outside the shell where it was defined. A shell function is a concept inside the shell. The parallel command itself has no way to access it.

在bash中调用export -f doit会通过环境导出函数，以便子进程将其拾取.但是只有bash可以理解bash的功能. (大型)*子bash进程可以调用它，但是不能调用其他程序，例如不能调用其他shell.

Calling export -f doit in bash exports the function via the environment so that it is picked up by child processes. But only bash understands bash functions. A (grand)*child bash process can call it, but not other programs, for example not other shells.

根据消息找不到命令"，看来您的首选shell是(t)csh.您需要告诉parallel来调用bash. parallel调用由SHELL环境变量¹指示的外壳，因此将其设置为指向bash.

Going by the message "Command not found", it appears that your preferred shell is (t)csh. You need to tell parallel to invoke bash instead. parallel invokes the shell indicated by the SHELL environment variable¹, so set it to point to bash.

export SHELL=$(type -p bash)
doit () { … }
export -f doit
parallel doit ::: 1 2 3

如果您只想为执行parallel命令而不是为脚本的其余部分设置SHELL:

If you only want to set SHELL for the execution of the parallel command and not for the rest of the script:

doit () { … }
export -f doit
SHELL=$(type -p bash) parallel doit ::: 1 2 3

我不确定如何处理远程作业，您可能需要通过 --env=SHELL 和--env=doit(请注意，这假定到bash的路径在所有地方都是相同的.)

I'm not sure how to deal with remote jobs, you may need to pass --env=SHELL in addition to --env=doit (note that this assumes that the path to bash is the same everywhere).

是的，应该在手册中更突出地提到这种奇怪之处. command参数的描述中有简短说明，但是它不是很明确(它应该解释command词是用空格作为分隔符，然后传递给$SHELL -c的)，并且SHELL甚至都没有列在环境变量部分. (我鼓励您将其报告为错误；我没有这样做，因为我几乎从未使用过该程序.)

And yes, this oddity should be mentioned more prominently in the manual. There's a brief note in the description of the command argument, but it isn't very explicit (it should explain that the command words are concatenated with a space as a separator and then passed to $SHELL -c), and SHELL isn't even listed in the environment variables section. (I encourage you to report this as a bug; I'm not doing it because I hardly ever use this program.)

¹

¹

这篇关于GNU Parallel和Bash函数:如何从手册中运行简单示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！