1.17.2.2 Build Scheduler

Algorithm

The algorithm is a subcomponent of the scheduler which is used to determine the input versions to the next build of a job. There are many factors that contribute to figuring out the next input versions. It can be anything that affects which resource versions will be used to schedule a build, such as version constraints or passed constraints in a get step, disabling versions through the web UI, etc. The algorithm can also fail to determine a successful set of input versions, which the error will be propogated to the preparation view in the build page.

If the algorithm computes a successful set of input versions, it will figure out whether or not the versions it computed can be used to produce a new build. This is done by comparing the triggerable input versions to the versions used by the previous build and if any of them have a different version, then the scheduler will know to schedule a new build. Conversly, if the input versions produced by the algorithm are the same as the previous build, then the scheduler will not create a new build.

Scheduling behavior

The scheduler will schedule a new build if any of the versions produced by the algorithm for trigger: true resources is not equal to the version used by the previous build of the job.

What this means is if the algorithm runs and computes an input version, the scheduler will create a new build as long as that version is different than the previous build's version for that same input. Even if that version has been used by a build 2 months ago, the scheduler will schedule a new build as long as it has not been used by the previous build.

If there are any input versions that are different than the previous build, it will trigger a new build. This can affect some users that are using the pinning functionality to run older versions.

Let's say you have a job that takes a resource as an input. It currently has two builds, build 1 has ran with version v1 of the resource and build 2 has ran with v2.

If I pin the input to an older version, a new build is produced because the pinned version v1 is not equivalent to the version of the previous build v2.

Then if I unpin the version, another build would be triggered again using the latest version.

This is because after unpinning the resource, the input version for the next build becomes the latest version v2 which is not equal to the version v1 used by the previous build.

This is to allow the current state of the builds to always reflect the current state of the versions. That being said, this behavior can be undesirable if the goal is to only run a job using an old version. In this case, build rerunning is the better choice. If you would like to learn more about build rerunning, you can read more about it in the build rerunning section.

Scheduling on demand

The scheduler runs on an interval, but rather than scheduling all the jobs within a deployment on every tick, it only schedules the jobs that need to be scheduled.

First, the scheduler determines which jobs need to be scheduled. Below are all the reasons why Concourse will think a job needs to be scheduled:

• Detecting new versions of a resource through a check

• Saving a new version through a put

• A build finishes for an upstream job (through passed constraints)

• Enabling/Disabling a resource version

• Pinning/Unpinning a resource version

• Setting a pipeline

• Updating a resource's resource_config

• Manually triggering a build

• Rerunning a build

• Multiple versions available for a version every constraint

Each job that is scheduled will use the algorithm to determine what inputs its next build should have. Then the build is scheduled and picked up by the Build Tracker.