set-pipeline: Configuring Pipelines

To submit a pipeline configuration to Concourse from a file on your local disk you can use the -c or --config flag, like so:

$ fly -t example set-pipeline --config pipeline.yml --pipeline my-pipeline

This will present a diff of the changes and ask you to confirm the changes. If you accept then Concourse's pipeline configuration will switch to the pipeline definition in the YAML file specified.

Using ((parameters))

The pipeline configuration can contain template variables in the form of ((foo-bar)). They will be replaced with YAML values populated by repeated --var or --load-vars-from flags.

This allows for credentials to be extracted from a pipeline config, making it safe to check in to a public repository or pass around.

For example, if you have a pipeline.yml as follows:

- name: private-repo
  type: git
    uri: git@...
    branch: master
    private_key: ((private-repo-key)) could then configure this pipeline like so:

$ fly -t example set-pipeline --pipeline my-pipeline --config pipeline.yml --var "private-repo-key=$(cat id_rsa)"

Or, if you had a credentials.yml as follows:

private-repo-key: |
  -----END RSA PRIVATE KEY----- could configure it like so:

$ fly -t example set-pipeline --pipeline my-pipeline --config pipeline.yml --load-vars-from credentials.yml

If both --var and --load-vars-from are specified, the --var flags take precedence.

Values other than strings (e.g. bools, arrays) may also be specified via params. Concatenation is also supported in the form of foo-((bar)).

Combining with get-pipeline

You can use process substitution to quickly replicate a pipeline by composing get-pipeline and set-pipeline together like so:

$ fly -t new set-pipeline -c <(fly -t old get-pipeline -p foo) -p foo

While this replication method may be useful for migrating pipelines between concourse instances, note the following caveats:

get-pipeline fetches the current pipeline configuration with variables already interpolated. So you are working around any previous parameterization.

Replicating a pipeline, unmodified, risks resource contention. For example, you could end up with two pipelines pushing to the same S3 bucket.

If you're starting a new pipeline, we recommend you try to work through the above examples before using this one-step clone.