问题描述
我使用 root 帐户在我的集群上安装了 Apache Airflow.我知道这是不好的做法，但这只是测试环境.我创建了一个简单的 DAG:
I installed Apache Airflow on my cluster using root account. I know it is bad practice, but it is only test environment. I created a simple DAG:
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta
dag = DAG('create_directory', description='simple create directory workflow', start_date=datetime(2017, 6, 1))
t1 = BashOperator(task_id='create_dir', bash_command='mkdir /tmp/airflow_dir_test', dag=dag)
t2 = BashOperator(task_id='create_file', bash_command='echo airflow_works > /tmp/airflow_dir_test/airflow.txt')
t2.set_upstream(t1)
问题是当我运行这个作业时，root 用户会执行它.我试图添加 owner 参数，但它不起作用.气流说:
The problem is that when I run this job, the root user executes it. I tried to add owner parameter, but it doesn't work. Airflow says:
Broken DAG: [/opt/airflow/dags/create_directory.py] name 'user1' is not defined
我的问题是，如何使用 root 以外的其他用户运行 Apache Airflow DAG?
My question is, how I can run Apache Airflow DAG using other user than root?
推荐答案
您可以使用 run_as_user 参数来模拟 unix 用户执行任何任务:
You can use the run_as_user parameter to impersonate a unix user for any task:
t1 = BashOperator(task_id='create_dir', bash_command='mkdir /tmp/airflow_dir_test', dag=dag, run_as_user='user1')
如果要将 default_args 应用到 DAG 中的每个任务，可以使用它:
You can use default_args if you want to apply it to every task in the DAG:
dag = DAG('create_directory', description='simple create directory workflow', start_date=datetime(2017, 6, 1), default_args={'run_as_user': 'user1'})
t1 = BashOperator(task_id='create_dir', bash_command='mkdir /tmp/airflow_dir_test', dag=dag)
t2 = BashOperator(task_id='create_file', bash_command='echo airflow_works > /tmp/airflow_dir_test/airflow.txt')
请注意，owner 参数用于其他用途，多租户.
Note that the owner parameter is for something else, multi-tenancy.
这篇关于如何以 Unix 用户身份运行 Apache Airflow DAG的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！