使用Nifi-Registry和Docker对Nifi流文件进行版本控制和持久化

问题描述

目前，我正在使用最新的稳定版本Nifi / Nifi-registry和Docker。我正在尝试在Gitlab中对流文件进行版本控制和持久化。
我在网络上找到了一些信息，但现在无法正常工作。所有组件都在工作，我也可以将文件从Nifi升级到Nifi注册，但是目前缺少Gitlab的最后一步。

currently I'm playing with the latest stable version Nifi/Nifi-registry and Docker. I'm trying to version and persist flowfiles in Gitlab.I found several information in the web but right now it's not working. All components are working and I'm also able to version files from Nifi to Nifi-Registry, but the last step to Gitlab is currently missing.

->我修改了并将providers.xml安装到容器中

--> I modified and mounted the providers.xml into the container

<flowPersistenceProvider>
  <class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
  <property name="Flow Storage Directory">./versioned_flows</property>
  <property name="Remote To Push">origin</property>
  <property name="Remote Access User">*Name*</property>
  <property name="Remote Access Password">*Token*</property>
</flowPersistenceProvider>

andybody是否有一些经验，也许还有代码片段？

Does andybody have some experiences and maybe a code snippet? I would appreciate that.

非常感谢。

亲切的问候，
T

Kind regards,T

推荐答案

尽管这是一篇老文章，但如果对任何人有帮助，我都会很高兴。

Although it is an old post, I will be happy if it is helpful to anyone.

我有以下docker文件夹&它同时运行nifi和它是注册表容器。

I have the following docker folder & it runs both nifi & it's registry containers.

复制nifi目录中的 conf文件夹（可以从不运行卷的容器中获取）

Copy the 'conf' folder inside nifi directory (can be taken from running the container without volume)

运行 docker卷创建nifi_data

docker-compose.yml文件为：

The docker-compose.yml file is:

version: "3.7"
services:

nifi:
  container_name: nifi
  image: apache/nifi:1.11.4
  ports:
    - target: 8080
      published: 8080
      protocol: tcp
      mode: host
  restart: always
  environment:
    - NIFI_WEB_HTTP_HOST=0.0.0.0
    - NIFI_JVM_HEAP_INIT=4g
    - NIFI_JVM_HEAP_MAX=6g
    - NIFI_VARIABLE_REGISTRY_PROPERTIES=./conf/env.properties
  volumes:
    - nifi_data:/opt/nifi/nifi-current/
    - ./nifi/extensions:/opt/nifi/nifi-current/extensions
    - ./nifi/conf:/opt/nifi/nifi-current/conf

nifi-registry:
  container_name: nifi-registry
  image: apache/nifi-registry:0.7.0
  ports:
    - target: 18080
      published: 18080
      protocol: tcp
      mode: host
  environment:
    - NIFI_REGISTRY_WEB_HTTP_HOST=0.0.0.0
    - JVM_OPTS=-Xmx512m -Xms512m -XX:MaxPermSize=1g
  volumes:
    - $PWD/registry/providers.xml:/opt/nifi-registry/nifi-registry-current/conf/providers.xml
    - $PWD/registry/flow-storage:/opt/nifi-registry/nifi-registry-current/flow_storage
    - $PWD/registry/database:/opt/nifi-registry/nifi-registry-current/database
volumes:
  nifi_data:
    external: true
networks:
  default:
    external:
      name: nifi-network

注意：属性（env.properties）&自定义处理器位于扩展目录下。另外，由于nifi在 nifi用户下运行，您可能会遇到权限问题-我相信您会把它整理出来：）

Note: I have custom properties (env.properties) & the custom processors are under 'extensions' directory. Also, you may have permissions issue since nifi runs under 'nifi' user - I am sure you will sort it out :)

providers.xml（采用默认值-注释文件系统）提供者&取消注释git提供者）

The providers.xml (take the default one - comment the filesystem provider & uncomment the git provider)

<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">./flow_storage</property>

最后是restart.sh文件：

And finally the restart.sh file:

chown -R 1000:1000 nifi registry // nifi's user:group
chmod -R 775 nifi registry
chmod -R g+s nifi registry
docker-compose down
docker-compose up -d

这篇关于使用Nifi-Registry和Docker对Nifi流文件进行版本控制和持久化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！