On this page:

2.4.4 Metrics

Metrics are essential in understanding how any large system is behaving and performing. Concourse can emit metrics about both the system health itself and about the builds that it is running. Operators can tap into these metrics in order to observe the health of the system.

In the spirit of openness, the metrics from our deployment are public. We consider it a bug to emit anything sensitive or secret into our metrics pipeline.

2.4.4.1 Configuring Metrics

The ATC can be configured to emit metrics on start.

Currently supported metrics emitters are InfluxDB, NewRelic, Prometheus, Datadog, and Riemann. There is also a dummy emitter that will just spit the metrics out in to the logs at DEBUG level, which can be enabled with the --emit-to-logs flag.

There are various flags for different emitters; run concourse web --help and look for "Metric Emitter" to see what's available.

2.4.4.2 What's emitted?

This reference section lists of all of the metrics that Concourse emits. We don't include the warning and critical levels as they will keep changing as we optimise the system. To find those, please refer to the source of truth: the code.

scheduling: full duration (ms)

This is the time taken (in milliseconds) to schedule an entire pipeline including the time taken to load the version information from the database and calculate the latest valid versions for each job.

Attributes

pipeline: The pipeline which was being scheduled.

scheduling: loading versions duration (ms)

This is the time taken (in milliseconds) to load the version information from the database.

Attributes

pipeline: The pipeline which was being scheduled.

scheduling: job duration (ms)

This is the time taken (in milliseconds) to calculate the set of valid input versions when scheduling a job. It is emitted once for each job per pipeline scheduling tick.

Attributes

pipeline: The pipeline which was being scheduled.
job: The job which was being scheduled.

worker containers

The number of containers that are currently running on your workers.

Attributes

worker: The name of the worker.

worker volumes

The number of volumes that are currently present on your workers.

Attributes

worker: The name of the worker.

build started

This event is emitted when a build starts. Its value is the build ID of the build. However, it is most useful for annotating your metrics with the start and end of different jobs.

Attributes

pipeline: The pipeline which contains the build being started.
job: The job which configured the build being started.
build_name: The name of the build being started. (Remember that build numbers in Concourse are actually names and are strings).
build_id: The ID of the build being started.

build finished

This event is emitted when a build ends. Its value is the duration of the build in milliseconds. You can use this metric in conjunction with build started to annotate your metrics with when builds started and stopped.

Attributes

pipeline: The pipeline which contains the build that finished.
job: The job which configured the build that finished.
build_name: The name of the build that finished. (Remember that build numbers in Concourse are actually names and are strings).
build_id: The ID of the build that finished.
build_status: The resulting status of the build; one of "succeeded", "failed", "errored", or "aborted".

http response time

This metric is emitted for each HTTP request to an ATC (both API and web requests). It contains the duration (in milliseconds) for each request and is useful for finding slow requests.

Attributes

route: The route which the HTTP request matched. i.e. /builds/:id
path: The literal path of the HTTP request. i.e. /builds/1234