Concourse is an open-source continuous thing-doer.

Built on the simple mechanics of resources, tasks, and jobs, Concourse presents a general approach to automation that makes it great for CI/CD.

Quick Start

Configure as code

resources:
- name: booklit
  type: git
  source: {uri: "https://github.com/vito/booklit"}

jobs:
- name: unit
  plan:
  - get: booklit
    trigger: true
  - task: test
    file: booklit/ci/test.yml

Visualize to verify

unit
booklit

A Concourse pipeline is like a distributed, continuous Makefile.

Each job has a build plan declaring the job's input resources and what to run with them when they change.

Your pipeline is then visualized in the web UI, taking only one click to get from a failed job to seeing why it failed.

The visualization provides a "gut check" feedback loop: if it looks wrong, it probably is wrong.

A more complicated example...

Jobs can depend on other jobs by configuring passed constraints. The resulting chain of jobs and resources is a dependency graph that continuously pushes your project forward, from source code to production.

unit rc ship
version
release
version
booklit
booklit
booklit
version
release-notes

This particular pipeline can be found in the Booklit repository.

CI under source control

$ fly -t ci set-pipeline -p booklit -c pipeline.yml
$ vim pipeline.yml
$ fly -t ci set-pipeline -p booklit -c pipeline.yml
$ git add pipeline.yml
$ git commit -m "initial pipeline"█

All configuration and administration is done using the fly CLI.

The fly set-pipeline command pushes the config up to Concourse. Once it looks good, you can then check the file in to source control. This makes it easy to recover your project if the Concourse server burns down.

Reproducible, debuggable builds

$ fly -t ci intercept -j booklit/unit -s unit
root@2c15ff11:/tmp/build/0df9eea0# ps
    PID TTY          TIME CMD
    171 pts/1    00:00:00 bash
   1876 pts/1    00:00:00 ps
root@2c15ff11:/tmp/build/0df9eea0# ls
depspath  gopath
root@2c15ff11:/tmp/build/0df9eea0# █

Everything runs in containers, ensuring a clean environment on every run.

Each task specifies its own image, giving it full control over its dependencies, rather than managing packages and state on your workers.

The fly intercept command will pop you right into one of your build's containers, making it easy to troubleshoot flaky builds.

Rapid local iteration

~/booklit $ fly -t ci execute -c ci/test.yml
executing build 1 at http://localhost:8080/builds/1
initializing
booklit: 4.74 MiB/s 0s
running gopath/src/github.com/vito/booklit/ci/test
fetching dependencies...
installing ginkgo...
running tests...
█

The fly execute command lets you run a build with local changes.

This build runs in exactly the same way as it would run in your pipeline, without having to push broken commits until it works.

When a build in the pipeline fails, you can run fly execute with the -j flag to run a one-off build with the same inputs as the failed build. You can then replace an input with your local changes with -i to see if your fix is valid.

Bring your own integrations

resource_types:
- name: rubygem
  type: registry-image
  source:
    repository: troykinsella/concourse-rubygems-resource

resources:
- name: rspec-gem
  type: rubygem
  source: {gem: rspec}

jobs:
- name: bundle
  plan:
  - get: rspec-gem
    trigger: true
  - # ...

Concourse does not have a complex plugin system. Instead, it focuses on a single strong abstraction: resource, which are implemented by resource types.

The pipeline.resources field configures external artifacts that your pipeline will monitor for changes, fetch from, and push to.

For example, a resource with type git refers to a git repository, which will be cloned in a get step and pushed to using a put step. Behind the scenes, Concourse will continuously run git fetch to look for new commits that jobs may want to trigger on.

At its core, Concourse knows nothing about git. It comes with a git resource type out of the box, but you could just as easily bring your own into your pipeline by setting the pipeline.resource_types field.

To see what resource types are available, check out the Resource Types catalog!