脚本 jenkinsfile 并行阶段

问题描述

我正在尝试使用 groovy DSL 编写脚本化的 Jenkinsfile，它将在一组阶段中包含并行步骤.

I am attempting to write a scripted Jenkinsfile using the groovy DSL which will have parallel steps within a set of stages.

这是我的詹金斯文件:

node {   
stage('Build') {
    sh 'echo "Build stage"'
}

stage('API Integration Tests') {
    parallel Database1APIIntegrationTest: {
        try {
            sh 'echo "Build Database1APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }               

    }, Database2APIIntegrationTest: {
        try {
            sh 'echo "Build Database2APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }

    }, Database3APIIntegrationTest: {
        try {
            sh 'echo "Build Database3APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }
    }
}

stage('System Tests') {
    parallel Database1APIIntegrationTest: {
        try {
            sh 'echo "Build Database1APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }               

    }, Database2APIIntegrationTest: {
        try {
            sh 'echo "Build Database2APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }

    }, Database3APIIntegrationTest: {
        try {
            sh 'echo "Build Database3APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }
    }
}
}

我希望有 3 个阶段:构建；集成测试和系统测试.在两个测试阶段中，我希望并行执行 3 组测试，每组针对不同的数据库.

I want to have 3 stages: Build; Integration Tests and System Tests.Within the two test stages, I want to have 3 sets of the tests executed in parallel, each one against a different database.

我有 3 个可用的执行者.一个在主服务器上，两个代理，我希望每个并行步骤在任何可用的执行器上运行.

I have 3 available executors. One on the master, and 2 agents and I want each parallel step to run on any available executor.

我注意到，在运行我的管道后，我只看到了 3 个阶段，每个阶段都标记为绿色.我不想查看该阶段的日志来确定该阶段中的任何并行步骤是否成功/不稳定/失败.

What I've noticed is that after running my pipeline, I only see the 3 stages, each marked out as green. I don't want to have to view the logs for that stage to determine whether any of the parallel steps within that stage were successful/unstable/failed.

我希望看到我的测试阶段中的 3 个步骤 - 标记为绿色、黄色或红色(成功、不稳定或失败).

I want to be seeing the 3 steps within my test stages - marked as either green, yellow or red (Success, unstable or failed).

我考虑过将测试扩展到他们自己的阶段，但我意识到不支持并行阶段(有人知道这是否会被支持吗?)，所以我不能这样做，因为管道也需要很长时间完成时间很长.

I've considered expanding the tests out into their own stages, but have realised that parallel stages are not supported (Does anyone know whether this will ever be supported?), so I cannot do this as the pipeline would take far too long to complete.

任何见解将不胜感激，谢谢

Any insight would be much appreciated, thanks

推荐答案

在 Jenkins 脚本化管道中，parallel(...) 采用 Map 描述要构建的每个阶段.因此，您可以预先以编程方式构建构建阶段，这种模式允许灵活的串行/并行切换.
我使用了与此类似的代码，其中 prepareBuildStages 返回一个 Map 列表，每个 List 元素按顺序执行，而 Map 描述了该点的并行阶段.

In Jenkins scripted pipeline, parallel(...) takes a Map describing each stage to be built. Therefore you can programatically construct your build stages up-front, a pattern which allows flexible serial/parallel switching.
I've used code similar to this where the prepareBuildStages returns a List of Maps, each List element is executed in sequence whilst the Map describes the parallel stages at that point.

// main script block
// could use eg. params.parallel build parameter to choose parallel/serial 
def runParallel = true
def buildStages

node('master') {
  stage('Initialise') {
    // Set up List<Map<String,Closure>> describing the builds
    buildStages = prepareBuildStages()
    println("Initialised pipeline.")
  }

  for (builds in buildStages) {
    if (runParallel) {
      parallel(builds)
    } else {
      // run serially (nb. Map is unordered! )
      for (build in builds.values()) {
        build.call()
      }
    }
  }

  stage('Finish') {
      println('Build complete.')
  }
}

// Create List of build stages to suit
def prepareBuildStages() {
  def buildStagesList = []

  for (i=1; i<5; i++) {
    def buildParallelMap = [:]
    for (name in [ 'one', 'two', 'three' ] ) {
      def n = "${name} ${i}"
      buildParallelMap.put(n, prepareOneBuildStage(n))
    }
    buildStagesList.add(buildParallelMap)
  }
  return buildStagesList
}

def prepareOneBuildStage(String name) {
  return {
    stage("Build stage:${name}") {
      println("Building ${name}")
      sh(script:'sleep 5', returnStatus:true)
    }
  }
}

生成的管道显示为:

并行块中可以嵌套的内容有一定的限制，请参阅 the管道文档 了解详细信息.不幸的是，尽管它不如脚本(恕我直言)灵活，但大部分参考资料似乎都偏向于声明式管道.管道示例页面是最有帮助的.

There are certain restrictions on what can be nested within a parallel block, refer to the pipeline documentation for exact details. Unfortunately much of the reference seems biased towards declarative pipeline, despite it being rather less flexible than scripted (IMHO). The pipeline examples page was the most helpful.

这篇关于脚本 jenkinsfile 并行阶段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！