我正在运行带有通过microk8s导入的自定义docker镜像的KubernetesPodOperator。在气流中运行DAG时,我会看到以下日志,直到收到超时消息AirflowException('Pod Launching failed: {error}'.format(error=ex))[2020-04-14 23:06:39,875] {logging_mixin.py:95} INFO - [[34m2020-04-14 23:06:39,875[0m] {[34mpod_launcher.py:[0m141} INFO[0m - Event: [1mk8s-pod-ml-3a541375[0m had an event of type [1mPending[0m[0m
搜索问题,我在ubuntus终端上运行microk8s kubectl get pods并得到:
kubernetes -  Airflow KubernetesPodOperator AirflowException ('Pod Launching failed: {error}' .format(error = ex))-LMLPHP

如您所见,在日志中,pod的名称几乎完全相同(开头处为1m除外),但状态为ErrImageNeverPull。除了this上面写着“KubernetesPodOperator的默认超时为120秒,这可能导致在下载较大的图像之前发生超时”,我在该状态中找不到与该状态或错误有关的任何内容。我将默认超时更改为600秒,但是没有用。

最佳答案

问题

Microk8s无法访问docker。我通过运行microk8s.inspect并找到

WARNING:  Docker is installed.
File "/etc/docker/daemon.json" does not exist.
You should create it and add the following lines:
{
    "insecure-registries" : ["localhost:32000"]
}

and then restart docker with: sudo systemctl restart docker
Building the report tarball
  Report tarball is at /var/snap/microk8s/1320/inspection-report-20200415_141500.tar.gz

在发出警告之后,所述气流设法将图像部署在 pods 中。

关于kubernetes - Airflow KubernetesPodOperator AirflowException ('Pod Launching failed: {error}' .format(error = ex)),我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/61218845/

10-15 21:54