The following tables manage objects exposed directly to users, either via the API or by configuring pipelines.
The pipelines table contains details about a pipeline; its name, which team it belongs to, whether it is paused, and whether it is public.
There are also a number of attributes which are used by the runtime:
The build scheduler does not schedule builds for jobs which are
The build scheduler does not schedule builds for jobs which have not yet had their
The build scheduler does not schedule builds for jobs that have had their build
The builds table tracks the details of every run of every job or one-off build, including its name, the status of the build, whether the build has been scheduled or completed, and information about the start and end times of the build.
An entry in the resources table represents a resource defined in a pipeline. The resources table has columns for configuration, name, type, and details about resource version checking.
This table contains versions of resources discovered by checking. Versioned resources have attributes describing their version, metadata, resource type, and whether the version is enabled in the pipeline.
An entry in the resource types table represents a custom resource type defined in a pipleine, and all of the information neccessary to pull the image bits for a custom resource type. This includes the resource type used to fetch the resource type's image (eiher a base_resource_type or another custom resource) , the version of the image to fetch using that resource, and the configuration of the custom resource type.
Concourse's database manages the abstractions around resources, resource configuration, resource versioning, and resource types. These tables help make the Resources concept operate as expected, scale across many pipelines, and to many different types.
A resource config is a distinct configuration of a resource type, not specific to any pipeline resource.
A resource config is used for discovering versions. Unlike pipeline resources, they have no name, and are identified entirely by their type and source. Resource configs are shared across pipelines and teams, since identical resource configurations can be reused.
A resource cache represents all information necessary to fetch a particular set of bits, i.e. the cache.
A resource cache always points to its
resource_configs, and also specifies the version and params.
resource_caches both point to each other. This loop is closed by a resource config that points to a base resource type at the end of the chain. This represents the case of a resource cache provided by a custom resource type, which in turn may be provided by another custom resource type, and so on, until they reach a base type.
A cache use is a join table between a resource cache and a 'user', which lets us know when the use is no longer needed. Think of it as a reference counter.
A use can be tied to a
builds, in which case the use can be removed when the build is finished. A cache is in use by a build when the build is fetching it as an input. This is to prevent the garbage collector from reaping caches that are no longer desired in the long run, but are still in use by a build.
A use can be tied to a
containers, in which case the use can be removed when the container goes away. A cache is in use by a container when it's being used as the container's image, either via
image_resource or a custom resource type.
A resource config check session is used for periodic resource checking of
resource_types by pointing to their resource config. There is only one check container per resource config, even if it's defined in many pipelines.
A check session's expiry is tied to its worker's uptime, as a simple way to force balancing of check containers across workers more quickly after a deploy.
A worker resource config check session is a join table tying a resource config check session to a worker base resource type. When either go away, the worker resource config check session goes away. This is so that containers using an old base resource type are reaped and recreated.
An entry in the base resource types table represents a resource type which can be provided by any worker. It is used to keep
resource_configs as general as possible, and not tied to particular workers. This in turn allows
resource_caches to also be general, as is desired for Concourse's higher-level pipeline resource caching semantics.
An entry in the worker base resource type table represents a location on a particular worker where the rootFS for a specific version of a base_resource_type is found. When a worker registers, its
base_resource_types are synced by first removing old versions and then inserting new ones.
A worker resource cache is automatically removed when its worker base resource type or resource cache is removed. This is used to automatically invalidate any caches that were fetched with an old version of a base resource type.
The containers table represents the set of all containers across all the workers, and keeps track of their state such that no container is ever left behind on a worker.
Containers have a handful of really important attributes:
The unique identifier of the container in Garden.
The name of the worker where the container is located.
Containers can be one of four types, each with individual references to their relating object.
This signifies that the container is dependant on another container which is busy checking for the image this container will be based on. This is used in the case of custom resource types, or tasks with image_resource. This container will be in the
CREATING state until the image is fetched later.
This signifies that the container is dependant on another container which is busy downloading the bits for the image this container will be based on. This is used in the case of custom resource types, or tasks with image_resource. This container will be in the
CREATING state until the image is fetched.
If this container is for a build step, this column refers to the
builds (id) this container is related to.
The volumes table represents the set of all volumes across all the workers, and keeps track of their state such that no volume is ever left behind on a worker.
Volumes have a handful of really important attributes.
The unique identifier of the volume in BaggageClaim.
The name of the worker where the volume located.
The container this volume is associated with.
If this volume is used for a worker base resource type this column points to it.
If this volume is for a worker resource cache this column points to it.
The workers table represents all of the workers that have registered with the Concourse cluster.
The unique name for the worker used to identify it in the cluster.
The full address of the Garden server running for the worker.
state: RUNNING, STALLED, LANDING, LANDED, RETIRING
The current state of the worker. This keeps track of the worker's lifecycle in order to handle the lifecycle of workers and their ability to have new containers or volumes created on them.
The full address of the worker's baggageclaim server; used to store volumes on the worker.