问题描述
我想通过sbatch_run.sh
脚本使用不同的参数多次运行python脚本main.py
,如下所示:
I wanted to run a python script main.py
multiple times with different arguments through a sbatch_run.sh
script as in:
#!/bin/bash
#SBATCH --job-name=sbatch_run
#SBATCH --array=1-1000
#SBATCH --exclude=node047
arg1=10 #arg to be change during runs
arg2=12 #arg to be change during runs
python main.py $arg1 $arg2
参数由sbatch在bash文件中编码.我担心如果我sbatch_run.sh
多次一个接一个运行,但是每次运行期间更改arg1和arg2的值,可能会导致我的运行出错.例如,如果我这样做:
The arguments are encoded in the bash file ran by sbatch. I was worried that if I ran sbatch_run.sh
multiple times one after the other but changing the value of arg1 and arg2 during each run, that it might cause errors in my runs. For example if I do:
sbatch sbatch_run.sh # with arg1=10 and arg2=12
,然后立即,在我更改了sbatch_run.sh
但再次按以下方式运行文件后:
and then immediately after I change sbatch_run.sh
but run the file again as in:
sbatch sbatch_run.sh # with arg1=69 and arg2=666
我的运行将全部以最后一个运行(即arg1=69
和arg2=666
)运行,而不是每个运行都有自己的参数.
would case my runs to all run with the last one (i.e. arg1=69
and arg2=666
) instead of each run with its own arguments.
我肯定知道,如果我对main.py中的参数进行硬编码,然后运行相同的sbatch脚本,但是更改main.py,它将运行最后一个脚本.我想知道是否也可以更改sbatch_run.sh
脚本.
I know for sure that if I hard code the arguments in main.py and then run the same sbatch script but change the main.py it will run the last one. I was wondering if that is the case too if I change the sbatch_run.sh
script.
您知道,我确实通过运行1000个脚本来尝试了该实验,然后将其中一些排队,并放置了sleep命令,然后更改了sbatch_run.sh
.似乎并没有改变我的跑步方式,但是,如果我错了,这太重要了,以至于无意间犯错,并想确保我也提出了要求.
Just so you know, I did try this experiment, by running 1000 scripts, then some get queued and put a sleep command and then change the sbatch_run.sh
. It seems to not change what my run is, however, if I am wrong this is way too important to be wrong by accident and wanted to make sure I asked too.
我记录下来:
#!/bin/bash
#SBATCH --job-name=ECHO
#SBATCH --array=1-1000
#SBATCH --exclude=node047
sleep 15
echo helloworld
echo 5
,然后将回声更改为回声10或回声byebyeworld.
and then change the echo to echo 10 or echo byebyeworld.
推荐答案
运行sbatch时,Slurm将提交脚本复制到其内部数据库中;您可以通过以下实验来说服自己:
When sbatch is run, Slurm copies the submission script to its internal database ; you can convince yourself with the following experiment:
$ cat submit.sh
#!/bin/bash
#SBATCH --hold
echo helloworld
--hold
用于确保作业没有开始.提交:
The --hold
is there to make sure the job does not start. Submit it :
$ sbatch submit.sh
然后修改提交脚本:
$ sed -i 's/hello/bye/' submit.sh
$ cat submit.sh
#!/bin/bash
#SBATCH --hold
echo byeworld
,现在使用control show job
查看Slurm计划运行的脚本:
and now use control show job
to see the script Slurm is planning to run:
$ scontrol show -ddd job YOURJOBID
JobId=******* JobName=submit.sh
[...]
BatchScript=
#!/bin/bash
#SBATCH --hold
echo helloworld
[...]
尽管原始脚本没有改变,但它没有改变.
It hasn't changed although the original script has.
Slurm的最新版本使用scontrol write batch_script
而不是scontrol show -dd job
来显示提交脚本.
Recent versions of Slurm use scontrol write batch_script
rather than scontrol show -dd job
to show the submission script.
这篇关于在运行过程中更改发送给Sbatch的bash脚本是一个坏主意吗?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!