Skip to content

Templating Across Multiple Files

Once a template grows past a handful of jobs, keeping everything in one file gets unwieldy. ytt lets you split a template across multiple files, then render them all together as if they were one.

The Files

template.yml — the entry point. It loads two library files and calls their functions to assemble the pipeline:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
#@ load("@ytt:data", "data")
#@ load("jobs.lib.yml", "unit_tests", "build_rc_image", "deploy")
#@ load("resources.lib.yml", "resources")

jobs:
- #@ unit_tests()
- #@ build_rc_image()
- #@ deploy("dev")
- #@ deploy("prod")

resources: #@ resources(data.values.branch_name, data.values.artifact_slug)

#! TODO: temp fix to ensure usage of the latest mock resource
resource_types:
- name: mock
  type: registry-image
  source:
    repository: concourse/mock-resource

jobs.lib.yml — a library file exporting one function per job. deploy() takes an environment name as an argument and branches its passed constraints depending on whether it's called for dev or prod:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
#@ def unit_tests():
name: unit-tests
build_log_retention:
  builds: 50
plan:
- get: repo
  trigger: true
- task: run-unit-tests
  config:
    platform: linux
    image_resource:
      type: mock
      source:
        mirror_self: true
    inputs:
    - name: repo
    run:
      path: sh
      args:
      - -c
      - |
        echo running the unit tests...
        cat repo/branch.txt
        sleep 4
        echo tests passed!
#@ end

#@ def build_rc_image():
name: build-image
build_log_retention:
  builds: 50
plan:
- get: repo
  trigger: true
  passed: [unit-tests]
- task: build-image
  config:
    platform: linux
    image_resource:
      type: mock
      source:
        mirror_self: true
    inputs:
    - name: repo
    outputs:
    - name: image
    run:
      path: sh
      args:
      - -c
      - |
        echo building the image...
        date +%Y-%m-%d > image/version
        sleep 2
        echo image built!
- put: image-rc
  params:
    file: image/version
#@ end

#@ def deploy(deployment_env):
name: #@ "deploy-" + deployment_env
plan:
- in_parallel:
  - get: repo
    passed:
    #@ if deployment_env == "dev":
    - build-image
    #@ elif deployment_env == "prod":
    - deploy-dev
    #@ end
  - get: image-rc
    passed:
    #@ if deployment_env == "dev":
    - build-image
    #@ elif deployment_env == "prod":
    - deploy-dev
    #@ end
- put: #@ deployment_env + "-env"
  params:
    file: image-rc/oci_image
#@ end

resources.lib.yml — a second library file, exporting a single function that builds the resources: block from two parameters:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#@ def resources(branch_name, artifact_slug):
- name: repo
  icon: git
  type: mock
  source:
    initial_version: #@ "repo-at-" + branch_name
    create_files:
      branch.txt: #@ branch_name

- name: image-rc
  icon: oci
  type: mock
  source:
    initial_version: #@ artifact_slug + "-rc"
    create_files:
      oci_image: #@ artifact_slug + "-rc"

- name: dev-env
  icon: wrench
  type: mock

- name: prod-env
  icon: cloud-check
  type: mock
#@ end

vars.yml — the data values passed into the functions above:

1
2
3
4
#@data/values
---
branch_name: "feature-1024"
artifact_slug: "ft1024"

Rendering

Unlike the single-file example, this one is rendered by pointing ytt at the whole directory rather than naming files individually, since template.yml loads its library files by relative path:

ytt -f ./ > rendered.yml

What Gets Rendered

The four function calls in template.yml expand into four concrete jobs, chained together by passed constraints, alongside the four resources built by resources():

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
resources:
  - name: repo
    icon: git
    type: mock
    source:
      initial_version: repo-at-feature-1024
      create_files:
        branch.txt: feature-1024
  - name: image-rc
    icon: oci
    type: mock
    source:
      initial_version: ft1024-rc
      create_files:
        oci_image: ft1024-rc
  - name: dev-env
    icon: wrench
    type: mock
  - name: prod-env
    icon: cloud-check
    type: mock

jobs:
  - name: unit-tests
    build_log_retention:
      builds: 50
    plan:
      - get: repo
        trigger: true
      - task: run-unit-tests
        config:
          platform: linux
          image_resource:
            type: mock
            source:
              mirror_self: true
          inputs:
            - name: repo
          run:
            path: sh
            args:
              - -c
              - |
                echo running the unit tests...
                cat repo/branch.txt
                sleep 4
                echo tests passed!

  - name: build-image
    build_log_retention:
      builds: 50
    plan:
      - get: repo
        trigger: true
        passed: [ unit-tests ]
      - task: build-image
        config:
          platform: linux
          image_resource:
            type: mock
            source:
              mirror_self: true
          inputs:
            - name: repo
          outputs:
            - name: image
          run:
            path: sh
            args:
              - -c
              - |
                echo building the image...
                date +%Y-%m-%d > image/version
                sleep 2
                echo image built!
      - put: image-rc
        params:
          file: image/version

  - name: deploy-dev
    plan:
      - in_parallel:
          - get: repo
            passed: [ build-image ]
          - get: image-rc
            passed: [ build-image ]
      - put: dev-env
        params:
          file: image-rc/oci_image

  - name: deploy-prod
    plan:
      - in_parallel:
          - get: repo
            passed: [ deploy-dev ]
          - get: image-rc
            passed: [ deploy-dev ]
      - put: prod-env
        params:
          file: image-rc/oci_image

resource_types:
  - name: mock
    type: registry-image
    source:
      repository: concourse/mock-resource