2018
Concourse Updates (July 3–6)
Concourse, beavers, poutine and maple syrup
Designing for Space in Concourse
This feature is still in the early stages of development. We are picking up from our previous sneak peak post by James Ma. If you have a comment and want to participate in the conversation please visit the issue on GitHub.
Concourse Pipeline Engineers have been asking for a better way to test variations and combinations of a resources in their pipelines. When pipelines are composed of resources that duplicate themselves to account for product versions (semver), branches of a repo, and different environments, the resulting pipeline can be quite difficult to navigate and edit.
Concourse Update (Jun 11–15)
This was a post-release week, so we spent a lot of time merging in new code from the Users track, fixing our pipelines, and working on some neglected issues. All in all a solid week’s worth of work! On to the update
How We Build Concourse
Building on some of our previous posts on the Concourse team mechanics1, I wanted to spend some time going over how we actually build Concourse.
Distributed Garbage Collection
Before diving into the weeds of Concourse architecture, let’s briefly take a look at container and volume lifecycles in Concourse. Going forward in this post I will refer to container and volume together as a “worker resource”. In the current architecture, there are multiple ways for worker resource creation to be triggered; like when jobs are started in pipelines or checking for a newer version of a resource. The ATC is responsible for managing worker resource lifecycle like creating, transitioning and destroying them. Workers are designed to be dumb and follow the orders issued by ATC. Removal of worker resource on worker and its reference (in ATC) is done as part of Garbage collection (GC) process in ATC.