使用 Nifi-Registry 和 Docker 对 Nifi 流文件进行版本控制和持久化

问题描述

目前我正在使用最新的稳定版本 Nifi/Nifi-registry 和 Docker.我正在尝试在 Gitlab 中对流文件进行版本化和持久化.我在网上找到了一些信息，但现在它不起作用.所有组件都在工作，我还可以将文件从 Nifi 版本化到 Nifi-Registry，但目前缺少 Gitlab 的最后一步.

currently I'm playing with the latest stable version Nifi/Nifi-registry and Docker. I'm trying to version and persist flowfiles in Gitlab.I found several information in the web but right now it's not working. All components are working and I'm also able to version files from Nifi to Nifi-Registry, but the last step to Gitlab is currently missing.

--> 我修改providers.xml并将其挂载到容器中

--> I modified and mounted the providers.xml into the container

<flowPersistenceProvider>
  <class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
  <property name="Flow Storage Directory">./versioned_flows</property>
  <property name="Remote To Push">origin</property>
  <property name="Remote Access User">*Name*</property>
  <property name="Remote Access Password">*Token*</property>
</flowPersistenceProvider>

andybody 是否有一些经验，也许还有代码片段?我会很感激的.

Does andybody have some experiences and maybe a code snippet? I would appreciate that.

非常感谢.

亲切的问候，T

推荐答案

虽然是老帖子，但如果对大家有帮助，我会很高兴的.

Although it is an old post, I will be happy if it is helpful to anyone.

我有以下 docker 文件夹 &它同时运行 nifi 和它是注册表容器.

I have the following docker folder & it runs both nifi & it's registry containers.

复制 nifi 目录中的 'conf' 文件夹(可以从运行没有卷的容器中获取)

Copy the 'conf' folder inside nifi directory (can be taken from running the container without volume)

运行 docker volume create nifi_data

docker-compose.yml 文件为:

The docker-compose.yml file is:

version: "3.7"
services:

nifi:
  container_name: nifi
  image: apache/nifi:1.11.4
  ports:
    - target: 8080
      published: 8080
      protocol: tcp
      mode: host
  restart: always
  environment:
    - NIFI_WEB_HTTP_HOST=0.0.0.0
    - NIFI_JVM_HEAP_INIT=4g
    - NIFI_JVM_HEAP_MAX=6g
    - NIFI_VARIABLE_REGISTRY_PROPERTIES=./conf/env.properties
  volumes:
    - nifi_data:/opt/nifi/nifi-current/
    - ./nifi/extensions:/opt/nifi/nifi-current/extensions
    - ./nifi/conf:/opt/nifi/nifi-current/conf

nifi-registry:
  container_name: nifi-registry
  image: apache/nifi-registry:0.7.0
  ports:
    - target: 18080
      published: 18080
      protocol: tcp
      mode: host
  environment:
    - NIFI_REGISTRY_WEB_HTTP_HOST=0.0.0.0
    - JVM_OPTS=-Xmx512m -Xms512m -XX:MaxPermSize=1g
  volumes:
    - $PWD/registry/providers.xml:/opt/nifi-registry/nifi-registry-current/conf/providers.xml
    - $PWD/registry/flow-storage:/opt/nifi-registry/nifi-registry-current/flow_storage
    - $PWD/registry/database:/opt/nifi-registry/nifi-registry-current/database
volumes:
  nifi_data:
    external: true
networks:
  default:
    external:
      name: nifi-network

注意: 我有自定义属性 (env.properties) &自定义处理器位于扩展"目录下.此外，由于 nifi 在 'nifi' 用户下运行，您可能会遇到权限问题 - 我相信您会解决的 :)

Note: I have custom properties (env.properties) & the custom processors are under 'extensions' directory. Also, you may have permissions issue since nifi runs under 'nifi' user - I am sure you will sort it out :)

providers.xml(取默认的 - 注释文件系统提供程序并取消注释 git 提供程序)

The providers.xml (take the default one - comment the filesystem provider & uncomment the git provider)

<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">./flow_storage</property>

最后是restart.sh文件:

And finally the restart.sh file:

chown -R 1000:1000 nifi registry // nifi's user:group
chmod -R 775 nifi registry
chmod -R g+s nifi registry
docker-compose down
docker-compose up -d

这篇关于使用 Nifi-Registry 和 Docker 对 Nifi 流文件进行版本控制和持久化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！