On this page:
# across_var schema
# across_var.var
# across_var.values
# across_var.max_in_flight
# across_var.fail_fast
# across
1.11.9.1.1 Limitations

1.11.9.1 across Step Modifier

Run a step multiple times with different combinations of variable values.

across is considered an experimental feature, and its syntax/semantics may change. To enable across for your deployment, you must set the feature flag CONCOURSE_ENABLE_ACROSS_STEP.

The across step can be combined with the load_var step, the set_pipeline step, and instanced pipelines to maintain a dynamically sized group of related pipelines.

More fields are also available for variable interpolation with the across step. See Across Step & Dynamic Vars for details.

Outputs from steps ran within the across step are not available to steps outside of the across step.

Contains a list of across_var schema.

across_var schema

The name of the variable that will be added to the "." var source. This variable will only be accessible in the scope of the step - each iteration of the step gets its own scope.

If a variable of the same name already exists in the parent scope, a warning will be printed.

The list of values that the var will iterate over when running the substep. If multiple vars are configured, all combinations of values across all vars will run.

The list of values may also be interpolated. For instance, you may use the load_var step to first load a list of value schema into a local var, and then iterate across that dynamic list of values.

The following across will run the task foo/build.yml for each package defined in foo/packages-to-build.json with Go 1.15 and 1.16.

plan:
- get: foo
- load_var: packages
  file: foo/packages-to-build.json
- across:
  - var: package
    values: ((.:packages))
  - var: go_version
    values: ['1.15', '1.16']
  task: build
  file: foo/build.yml
  vars:
    go_version: ((.:go_version))
    package: ((.:package))

Supposing foo/packages-to-build.json had the following content: 

["./cmd/first", "./cmd/second", "./cmd/third"]

...then the task foo/build.yml would be run with the following var combinations:

  • {package: "./cmd/first", go_version: "1.15"}

  • {package: "./cmd/first", go_version: "1.16"}

  • {package: "./cmd/second", go_version: "1.15"}

  • {package: "./cmd/second", go_version: "1.16"}

  • {package: "./cmd/third", go_version: "1.15"}

  • {package: "./cmd/third", go_version: "1.16"}

Default 1. If set to all, the substep will run with all combinations of the current var in parallel. If set to a number schema, only that number of substeps may run in parallel.

If multiple vars are configured, the effective max_in_flight is multiplicative. For instance:

plan:
- across:
  - var: var1
    values: [a, b, c]
    max_in_flight: all
  - var: var2
    values: [1, 2]
  - var: var3
    values: [foo, bar]
    max_in_flight: 2

Here, 6 substeps will run in parallel, since all 3 of var1's values can run in parallel, and 2 of var3's values can run in parallel.

Default false. When enabled, the across step will fail fast by returning as soon as any sub-step fails. This means that running steps will be interrupted and pending steps will no longer be scheduled.

jobs:
- name: job
  plan:
  - across:
    - var: some-text
      values: ["hello-world", "hello-concourse"]
    task: running-((.:some-text))
    config:
      platform: linux
      image_resource:
        type: mock
        source:
          mirror_self: true
      run:
        path: echo
        args: ["((.:some-text))"]
resources:
- name: ci
  type: git
  source:
    uri: https://github.com/concourse/examples.git

jobs:
- name: job
  plan:
  - get: ci
  - across:
    - var: pipeline
      values: ["hello-world", "time-triggered"]
    do:
      - task: running-((.:pipeline))
        input_mapping:
          ((.:pipeline)): ci
        output_mapping:
          ((.:pipeline)): newci
        config:
          platform: linux
          image_resource:
            type: mock
            source:
              mirror_self: true
          inputs:
            - name: ((.:pipeline))
          outputs:
            - name: ((.:pipeline))
          run:
            path: cat
            args: ["((.:pipeline))/pipelines/((.:pipeline)).yml"]
      - task: newci-((.:pipeline))
        config:
          platform: linux
          image_resource:
            type: mock
            source:
              mirror_self: true
          inputs:
            - name: newci
          run:
            path: cat
            args: ["newci/pipelines/((.:pipeline)).yml"]
resources:
- name: ci
  type: git
  source:
    uri: https://github.com/concourse/examples.git

jobs:
- name: job
  plan:
  - get: ci
  - across:
    - var: pipeline
      values: ["hello-world", "time-triggered"]
    set_pipeline: ((.:pipeline))
    file: ci/pipelines/((.:pipeline)).yml

Use the do step to across over multiple steps. 

jobs:
- name: job
  plan:
  - across:
    - var: name
      values: ["Kaladin", "Jasnah"]
    do:  # takes a list of steps
    - task: saying-hello
      config:
        platform: linux
        image_resource:
          type: mock
          source:
            mirror_self: true
        run:
          path: echo
          args: ["Hello ((.:name))!"]
    - task: saying-bye
      config:
        platform: linux
        image_resource:
          type: mock
          source:
            mirror_self: true
        run:
          path: echo
          args: ["Bye ((.:name))!"]

You can use the across step to set a pipeline for each branch in a git repository.

plan:
- get: release-branches
  trigger: true
- get: ci
- load_var: branches
  file: release-branches/branches.json
- across:
  - var: branch
    values: ((.:branches))
  set_pipeline: release
  file: ci/pipelines/release.yml
  instance_vars: {branch: ((.:branch.name))}

When a new branch is added, a new pipeline will be created. When a branch is deleted, the pipeline will be automatically archived as described in the set_pipeline step.

For a more complete example, refer to Multi-Branch Workflows.

Limitations

The across step does not work with the get step or put step. The names of resouces are not interpolated within across steps. Trying to do the following will not work. 

- across:
  - var: version
    values: ["1.16", "1.17"]
  do:
    - get: go-((.:version))
    # or this
    - get: golang
      resource: go-((.version))

The main reason this does not work is that Concourse determines the inputs for a job before the job starts. Concourse has no way of determining inputs for a job while it's in the middle of running.

Current pipeline valdiation logic will also block you from setting the pipeline at all since Concourse validates the relationship between all resources and jobs by looking at get and put steps.

The above example will return an error like this when trying to set the pipeline: 

invalid jobs:
  jobs.job.plan.do[0].across.get(go): unknown resource 'go-((.:version))'