Concourse

5.4 Pipelines

A pipeline is the result of configuring Jobs and Resources together. When you configure a pipeline, it takes on a life of its own, to continuously detect resource versions and automatically queue new builds for jobs as they have new available inputs.

Pipelines are configured as declarative YAML files, fitting the following schema:

jobs: [job]

A set of jobs for the pipeline to continuously check.

resources: [resource]

A set of resources for the pipeline to continuously check.

resource_types: [resource_type]

A set of resource types for resources within the pipeline to use.

groups: [group]

A list of groups to use for cleaning up/organizing jobs in the web UI.

  1. 5.4.1 Setting Pipelines
    1. 5.4.1.1 fly set-pipeline
    2. 5.4.1.2 fly validate-pipeline
    3. 5.4.1.3 fly format-pipeline
  2. 5.4.2 Managing Pipelines
    1. 5.4.2.1 fly pipelines
    2. 5.4.2.2 fly rename-pipeline
    3. 5.4.2.3 fly pause-pipeline
    4. 5.4.2.4 fly unpause-pipeline
    5. 5.4.2.5 fly expose-pipeline
    6. 5.4.2.6 fly hide-pipeline
    7. 5.4.2.7 fly get-pipeline
    8. 5.4.2.8 fly destroy-pipeline
    9. 5.4.2.9 fly order-pipelines
  3. 5.4.3 Grouping Jobs

Examples

Hello, world!

The following example makes use of an embedded task config to define the smallest possible pipeline.

jobs:
- name: hello-world
  plan:
  - task: say-hello
    config:
      platform: linux
      image_resource:
        type: docker-image
        source: {repository: alpine}
      run:
        path: echo
        args: ["Hello, world!"]

Small Go Project

When it comes to real-world projects, task configs are usually stored alongside the code that it's testing. This makes the pipeline a bit smaller and makes the config able to change without needing to reconfigure the pipeline.

---
resources:
- name: booklit
  type: git
  source:
    uri: https://github.com/vito/booklit
    branch: master

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: unit
    file: booklit/ci/test.yml

Chaining Jobs

Promoting resources to downstream jobs is done by setting passed on a get step.

Note that nothing in unit says anything about triggering build. Job definitions are self-contained; they describe their dependencies and where they come from, which results in a dependency flow that Concourse pushes forward.

---
resources:
- name: booklit
  type: git
  source:
    uri: https://github.com/vito/booklit
    branch: master

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: unit
    file: booklit/ci/test.yml

- name: build
  plan:
  - get: booklit
    passed: [unit]
    trigger: true
  - task: unit
    file: booklit/ci/build.yml

Using Resource Types

Resource Types can be used to extend the functionality of your pipeline and provide deeper integrations. This example uses one to trigger a job whenever a new Dinosaur Comic is out.

---
resource_types:
- name: rss
  type: docker-image
  source:
    repository: suhlig/concourse-rss-resource
    tag: latest

resources:
- name: booklit-releases
  type: rss
  source:
    url: http://www.qwantz.com/rssfeed.php

jobs:
- name: announce
  plan:
  - get: booklit-releases
    trigger: true