1.12 Tasks

Running tasks with fly execute

One of the most common use cases of fly is taking a local project on your computer and setting it up with a task configuration to be run inside a container in Concourse. This is useful to build Linux projects on OS X or to avoid all of those debugging commits when something is configured differently between your local and remote setup.

You can execute a task like this:

$ fly -t example execute --config tests.yml

Your files will be uploaded and the task will be executed with them. The working directory name will be used as the input name. If they do not match, you must specify -i name=. instead, where name is the input name from the task configuration.

Fly will automatically capture SIGINT and SIGTERM and abort the build when received. This allows it to be transparently composed with other toolchains.

By default, fly execute will not send extra files or large files in your current directory that would normally be ignored by your version control system. You can use the --include-ignored flag in order to send ignored files to Concourse along with those that are not ignored.

If your task needs to run as root, then you can specify the -p or --privileged flag.

$ fly -t example execute \
    --config build-stemcell.yml \
    --input code=. \
    --input stemcells=../stemcells

inputs:
- name: code
- name: stemcells

$ fly -t example execute \
    --config task.yml \
    --inputs-from main/integration \
    --input foo=./foo

resources:
- name: my-repo
  type: git
  source: {uri: https://example.com}

- name: some-image
  type: registry-image
  source: {repository: ubuntu}

jobs:
- name: integration
  plan:
  - get: my-repo
  - get: some-image
  - task: my-task
    file: my-repo/task.yml
    image: some-image

$ fly -t example execute \
    --config task.yml \
    --inputs-from main/integration \
    --image some-image

$ fly -t example execute \
    --config build-stemcell.yml \
    --input code=. \
    --output stemcell=/tmp/stemcell

outputs:
- name: stemcell

params:
  FOO: fizzbuzz
  BAR:

BAR=hello fly execute

fly -t example execute --config tests.yml \
  -l vars.yml \
  -v some_string="Hello World!" \
  -y some_bool=true

fly -t example execute --config task.yml --tag bar

Task runtime environment

A task runs in a new container every time, using the image provided by task-config.image_resource as its base filesystem (i.e. /).

The command specified by task-config.run will be executed in a working directory containing each of the task-config.inputs. If any input is missing, the task will not run (and the container will not even be created).

The working directory will also contain empty directories for each of the task-config.outputs. The task must place artifacts in the output directories for them to be exported. This meshes well with build tools with configurable destination paths.

If your build tools don't support output paths, you can configure an input and output with the same path. The directory will be populated by the input, and any changes made to the directory will propagate downstream as an output.

Any task step params configured will be set in the environment for the task's command, along with any environment variables provided by the task's image (i.e. ENV rules from your Dockerfile).

The user the command runs as is determined by the image. If you're using a Docker image, this will be the user set by a USER rule in your Dockerfile, or root, if not specified.

Another relevant bit of configuration is task step privileged, which determines whether the user the task runs as will have full privileges (primarily when running as root). This is intentionally not configurable by the task itself, to prevent privilege escalation by submitting pull requests to repositories that contain task configs.

Putting all this together, the following task config:

---
platform: linux

image_resource:
  type: registry-image
  source:
    repository: golang
    tag: '1.6'

params:
  SOME_PARAM: some-default-value

inputs:
- name: some-input
- name: some-input-with-custom-path
  path: some/custom/path

outputs:
- name: some-output

run:
  path: sh
  args:
  - -exc
  - |
    whoami
    env
    go version
    find .
    touch some-output/my-built-artifact

...will produce the following output:

+ whoami
root
+ env
USER=root
HOME=/root
GOLANG_DOWNLOAD_SHA256=5470eac05d273c74ff8bac7bef5bad0b5abbd1c4052efbdbc8db45332e836b0b
PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
GOPATH=/go
PWD=/tmp/build/e55deab7
GOLANG_DOWNLOAD_URL=https://golang.org/dl/go1.6.linux-amd64.tar.gz
GOLANG_VERSION=1.6
SOME_PARAM=some-default-value
+ go version
go version go1.6 linux/amd64
+ find .
.
./some-input
./some-input/foo
./some
./some/custom
./some/custom/path
./some/custom/path/bar
./some-output
+ touch some-output/my-built-artifact

...and propagate my-built-artifact to any later task steps or put steps that reference the some-output artifact, in the same way that this task had some-input as an input.