`set_pipeline` Step

Configures a pipeline. Expand each section below for more details and examples.

Pipelines configured with the set_pipeline step are connected to the job that configured them and will be automatically archived in the following scenarios:

When the job that previously set a pipeline runs a successful build which did not configure the pipeline (i.e. the set_pipeline step was removed for that specific pipeline).
When the job is removed from its pipeline configuration (see job.old_name for renaming instead of removing).
When the job's pipeline is archived or destroyed.

This means any job that uses set_pipeline should set all still-desired pipelines in each build, rather than setting them one-by-one through many builds.

See fly archive-pipeline for what happens when a pipeline is archived.

set_pipline: identifier | self (required)

The identifier specifies the name of the pipeline to configure. Unless set_pipeline step team is set, it will be configured within the current team and be created unpaused. If set to self, the current pipeline will update its own config.

One pipeline configuring another

This is a way to ensure a pipeline stays up to date with its definition in a source code repository, eliminating the need to manually run fly set-pipeline.

jobs:
  - name: set-pipeline
    plan:
      - get: examples
        trigger: true
      - set_pipeline: hello-world  # pipeline's name
        file: examples/pipelines/hello-world.yml  # pipeline's config

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

file: file-path (required)

The path to the pipeline's configuration file.

file points at a .yml file containing the pipeline configuration, which allows this to be tracked with your resources or generated by a task step.

The first segment in the path should refer to another artifact from the plan, and the rest of the path is relative to that artifact.

Fetching and configuring a pipeline

The get step can be used to fetch your configuration from a git repo and autoconfigure it using a set_pipeline step:

jobs:
  - name: set-pipeline
    plan:
      - get: examples
        trigger: true
      - set_pipeline: hello-world  # pipeline's name
        file: examples/pipelines/hello-world.yml  # pipeline's config

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

instance_vars: vars

A map of instance vars used to identify instanced pipelines. These vars will also be interpolated into the pipeline config.

Note

Variables set with this field will not propagate to tasks configured via task step file. If you want those variables to be determined at the time the pipeline is set, use task step vars as well.

Info

Instance pipelines are not enabled by default if your Concourse is on a version <v8. To enable this feature, set the --enable-pipeline-instances flag or env var CONCOURSE_ENABLE_PIPELINE_INSTANCES to true on the web node.

Configuring instance vars

The following pipeline will create one instance group with three pipelines. The instance group is called my-bots and each pipeline has a different set of instance_vars making it distinct from the other pipelines in the instance group.

jobs:
  - name: set-pipeline-instance-group
    plan:
      - get: examples
      - in_parallel:
          - set_pipeline: my-bots
            file: examples/pipelines/pipeline-vars.yml
            instance_vars:
              first: initial
              number: "9000"
              hello: HAL
          - set_pipeline: my-bots
            file: examples/pipelines/pipeline-vars.yml
            instance_vars:
              first: second
              number: "3000"
              hello: WALLY-E
          - set_pipeline: my-bots
            file: examples/pipelines/pipeline-vars.yml
            instance_vars:
              first: the-third
              number: "6000"
              hello: R2D2

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

Configuring instance vars and vars

Both instance_vars and vars may be statically. The difference between the two fields is that instance_vars are used to identify a pipeline and render the pipeline config. vars are only used for rendering the pipeline config:

jobs:
  - name: set-pipeline-vars-and-instance-vars
    plan:
      - get: examples
      - set_pipeline: my-bots
        file: examples/pipelines/pipeline-vars.yml
        instance_vars:
          first: initial
          number: "9000"
        vars:
          hello: HAL

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

vars: vars

A map of template variables to pass to the pipeline config. Unlike instance_vars, vars are solely used to for interpolation, and do not become a part of the pipeline's identifier.

Note

Variables set with this field will not propagate to tasks configured via task step file. If you want those variables to be determined at the time the pipeline is set, use task step vars as well.

Configuring static vars

jobs:
  - name: set-pipeline-vars-only
    plan:
      - get: examples
      - set_pipeline: pipeline-set-with-vars
        file: examples/pipelines/pipeline-vars.yml
        vars:
          first: initial
          number: "9000"
          hello: HAL

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

vars_files: file-path

A list of paths to .yml files that will be passed to the pipeline config in the same manner as the --load-vars-from flag to fly set-pipeline. This means that if a variable appears in multiple files, the value from a file that is passed later in the list will override the values from files earlier in the list.

Configuring static vars with a vars file

Where the vars file looks like:

first: initial
number: "9000"
hello: HAL

And the pipeline config is:

jobs:
  - name: set-pipeline-vars-only
    plan:
      - get: examples
      - set_pipeline: pipeline-set-with-vars
        file: examples/pipelines/pipeline-vars.yml
        var_files:
          - examples/pipelines/vars-file.yml

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

team: identifier

By default, the set_pipeline step sets the pipeline for the same team that is running the build.

The team attribute can be used to specify another team.

Only the main team is allowed to set another team's pipeline. Any team other than the main team using the team attribute will error, unless they reference their own team.

Setting a pipeline on another team

jobs:
  - name: set-pipeline
    plan:
      - get: examples
        trigger: true
      - set_pipeline: hello-world
        file: examples/pipelines/hello-world.yml
        team: other-team  # name of the team goes here

resources:
  - name: examples
    type: git
    icon: github
    source:
      uri: https://github.com/concourse/examples.git

set_pipeline Step

`set_pipeline` Step