The Concourse 2018 Community Survey is here and we'd love your feedback!

Concourse

2.1 Download

If you are currently on a version older than v3.6.0, you must first upgrade to v3.6.0 before upgrading past it!

lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:4.2.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v4.2.1
garden-runc release: v1.16.3

See the Concourse BOSH Deployment repo for documentation.

latest

v4.2.1

September 17, 2018
fix
security

Fixed a bug in the access checking logic for a few worker-related API endpoints. This was introduced in v4.0.0.

Specifically, the API allowed any logged-in user to prune, retire, and land workers, in addition to performing a few harmless internal garbage-collection calls.

Thankfully, the impact of this is fairly small, as the worst someone can do is make your workers become landing or retiring. Which is annoying, but at least there is no risk of gaining access to any of your workers or sensitive pipeline data.

outdated

v4.2.0

September 17, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:4.2.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v4.2.0
garden-runc release: v1.16.3

See the Concourse BOSH Deployment repo for documentation.

vape naysh

fix
security

Fixed a potential information leak: when logged in and viewing a resource from some other team's (exposed) pipeline, you can no longer view the resource's check error, as it may unintentionally have sensitive info in the output. This regressed in v4.0.0.

feature

The dashboard view will now indicate whether you are a member of each team or whether you're only seeing it because it has exposed pipelines.

feature

The web node can now be configured to periodically emit build logs to a syslog endpoint! This is configured via --syslog-X flags on the concourse web command. When enabled, build logs will be shipped off in batch as builds complete.

fix
breaking

In v4.1.0 we accidentally broke support for --postgres-data-source. This flag has been deprecated ever since v2.7.2 (over a year ago), so we've opted to finally remove it.

fix

When we fixed the login bug in v4.1.0 by storing Dex state in the database (rather than in-memory), that effectively made it so that any changes made to auth settings (like local user config, GitHub config, etc) would not take effect.

This was because prior to the fix we were using an in-memory store, so all we had to do before was create all the configs anew, but now that things persist we have to do a comparison and update/remove things that were changed or removed from the flags. Thankfully @edtan noticed this and fixed it in a PR!

fix

With switching to Dex for auth in v4.0.0 we ended up using the external URL as part of the internal login flow callbacks. This meant it would break if your external URL could not be reached (perhaps it's behind a reverse proxy with auth, or a self-signed-cert, or a firewall).

This is now fixed - the callbacks will go to the internal address only. Sorry for the turbulence! A few folks were stuck on this.

fix

The fly intercept command will no longer list containers that are still being created and are not yet interceptible, which would lead to an unhelpful websocket: bad handshake error.

fix

Fixed one more instance where logged-in users would get logged out too soon. Specifically, on first login the cookie would expire in 1 hour rather than 24 hours.

The root cause of this was silly.

fix

Fixed a potential panic in the 'delete worker' API endpoint, which is used internally as part of the worker draining lifecycle.

fix

The BOSH release now respects the configured postgresql.client_cert property, thanks to a fix by @flavorjones. This broke back in v3.3.0 when we tweaked the type of the property.

fix

Removed an artificial limit to the garbage collector that was originally to prevent a stampede of work on a single worker. Now that workers garbage-collect themselves, this was no longer necessary, and only slowed down the database side of the garbage collection lifecycle.

fix

Cleaned up our idempotent process reattaching mumbo-jumbo to not rely on Garden properties which should help quiet down the worker node logs (from when we check for a property that hasn't yet been set).

fix

Fixed a bug that caused the Vault login retry logic to go into a fast loop if retrying failed for long enough to exceed the maximum retry backoff. Thanks for the PR, @edtan!

outdated

v4.1.0

August 30, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:4.1.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v4.1.0
garden-runc release: v1.16.3

See the Concourse BOSH Deployment repo for documentation.

This release unintentionally broke --postgres-data-source, which was deprecated back in v2.7.2. We're going to fully remove it in the next release, so now's a good time to switch to the new flags!

fix

Fixed an annoying issue affecting deployments with multiple web nodes. We were configuring an in-memory store for the Dex auth flow, meaning your login session would only work on one ATC at a time, and could fail partway through the redirect dance.

We now store this state in the same Postgres database that you're already configuring today, so everything should "just work" from here on.

feature

Task caches can now be cleared via fly clear-task-cache, thanks to a few PRs by @edtan! This is handy when something has gone terribly wrong with your cache and you need to reset everything. It happens.

feature

Workers can now be registered with a --ephemeral flag. When specified, the worker will be immediately removed once it stalls.

This is useful for situations where you don't have careful control over when and how the worker goes away, for example with preemptible machines or when running with Docker Compose or on a development machine.

Thanks to @tanner-bruce for kicking off the PR for this!

breaking

We've removed the Bosh Deployment resource from our core set of resources, as it has been deprecated for over a year. Use the CloudFoundry BOSH deployment resource instead! It's much better.

feature

You can now force an immediate check of a resource type via fly check-resource-type. This should help shorten feedback loops when testing your own resources types.

Note that the need for this may soon go away if we follow through with RFC #8 which proposes merging Resource Types into Resources.

fix

The concourse/concourse Docker image now contains the file command, which is useful for...working. Without it the btrfs volume driver setup would fail. Sorry about that, and thanks to @ElfoLiNk for submitting the PR!

fix

The --external-url and --peer-url flags for concourse web will now infer defaults that match the configured --bind-ip and --bind-port, rather than blindly defaulting to http://127.0.0.1:8080 and breaking when the bind IP/port are changed.

fix

Inputs that are "new" will now have a yellow icon. This replaces the input background highlighting that was accidentally removed with 4.0's recoloring. We opted to make the icon yellow instead of highlighting the background as we were quickly approaching 50 Shades of Grey.

fix

We've fixed a scary container snowballing failure mode that could happen when check containers failed to create. This was a somewhat hairy bug; see concourse #2454 for more information.

feature

Resources can now be pinned across the pipeline as part of the pipeline config by specifying version on the resource definition in the pipeline. This is analogous to setting version on every get step that references the resource.

feature

Pipeline credentials can now be verified via a new --check-creds flag available on fly set-pipeline. This will simply try fetching all of them from the configured credential manager, and let you know which ones couldn't be interpolated. Thanks for the PRs, @edtan!

feature

The Git resource is now smart enough to handle shallow clones while still being able to fetch commits that would not normally be included by the configured depth. This should make configuring depth: 1 safe, so we've removed the condescending warning from the README. Thanks for the PR, @norbertbuchmueller!

feature

The ATC now exposes an API endpoint for performing a health-check against the configured credential manager, at /api/v1/info/creds. It'll propagate whatever information may be useful, depending on your credential manager backend. Note that this endpoint is only accessible by admin users (members of the main team).

feature

When logging in to fly, if you're already logged in via the web UI it'll just shimmy the existing token over to fly rather than requiring you to go through the login flow all over again.

This also fixes the annoying behavior of having to log back in to the web UI whenever logging in to fly. Huzzah!

feature

We gave up on using third-party Go migration libraries and wrote our own. This should make failed migrations a lot easier to troubleshoot and recover from. All of our migrations run in transactions, so there's no more confusing "dirty" state, and failed migrations will record the failure error in the database.

We'll be extracting this package from the ATC soon as a proper library.

feature

Previously Concourse would allow you to configure arbitrary params in a pipeline, even if the task file itself didn't configure them. This was confusing as it meant the tasks could not be trusted to describe all their required parameters.

Concourse will now emit a warning to the task logs upon noticing this. A future release will turn this into an error. Thanks for the PR, @edtan!

feature

Searching on the dashboard will now live-update the URL bar, making it easy to copy-paste and share the search with your best friends. Thanks for the PR, @SHyx0rmZ!

fix

Previously, when searching something on the dashboard that filtered out all the pipelines for a given team, the UI showed messaging that made it look like the team had no pipelines at all, when in reality they just didn't match your search terms. Now the UI will just hide the teams entirely!

feature

The fly validate-pipeline can now be instructed to print the interpolated pipeline config via --output. Thanks for the PR, @henderjm!

feature

The BOSH release now exposes properties for configuring LDAP auth, thanks to a PR by @JamesClonk!

fix
breaking

All our auth CA certificate properties were broken - they were type: certificate but didn't actually pluck the certificate part off of the property. They're now fixed thanks to a PR by @ArthurHlt.

If this was working for you before, you'll have to change your manifest such that the properties specified conform to the BOSH certificate type - so you'll just need to take the existing value and nest it under certificate:.

fix

The TSA will now respect the configured log level for worker heartbeating logs, thanks to a PR by @edtan.

fix

We got rid of a ton of annoying and chatty logs from the TSA:

  • closing-channel

  • closing-forwarded-tcpip

  • waiting-for-tcpip-io

  • done-waiting

These were useful a long while back as it's fairly tricky to implement a TCP/IP-forwarding SSH server. But now it pretty much works and it was like printing a long line for breathing in and another for breathing out.

fix

The Docker Image resource will now fail more betterly when the build_args_file can't be parsed. Thanks @ghostsquad!

fix

The Docker Image resource will now forego starting the Docker daemon if skip_download: true is set. Thanks @norbertbuchmueller!

fix

@petrosagg a few places where Concourse couldn't compile on 32-bit platforms. See concourse #1379 for more information!

feature

The Bosh Io Release resource now supports configuring a version regexp by which to filter detected versions, thanks to a PR by @dark5un!

feature

The OIDC auth method now supports being configured with a whitelist of Google hosted domains, thanks to a PR by @rubenv!

feature

The search field on the dashboard will now live-update the URL, making it easy to share and bookmark pre-set filters. Thanks for the PR, @SHyx0rmZ!

fix

The S3 resource can now be used with Dell's EMC ECS object store, thanks to a fix by @adam-power!

fix

The Docker Image resource will now fail with a clearer error when your ECR credentials are incorrect, thanks to a PR by @GrantSheehan!

feature

The Docker Image resource now supports interpolating the Concourse-provided env vars in build args, thanks to a PR by @norbertbuchmueller!

feature

The Git resource is now rocking the latest and greatest version of Git LFS, thanks to a PR by @alucillo!

fix

We went ahead and started using ON CONFLICT in more places where we were sorely needing safe upsert mechanics prior to our bump to Postgres 9.5. Database integrity is cool and deserves to be release notes! Don't @ me.

fix

Building a precompiled BOSH release has been fixed. One of our Windows packages was missing the exiter.ps1 short-circuit in its spec. Thanks for the PR, @RomRider!

feature

The Git resource will now emit a short SHA to .git/short_ref, which can be useful for dynamic tagging and such. Thanks for the PR, @suda!

feature

The S3 resource now supports skip_download: true in params.

feature

The Cf resource now supports configuring vars and vars_files, thanks to a PR by @jmcarp!

feature

The Cf resource now configures env under each application in the manifest, rather than at the top level (which is deprecated). Thanks for the PR, @jmcarp!

fix

Clarified the help-text for local user configuration to mention that the password can be in plaintext, and if bcrypted it must have a minimum cost of 10.

fix

Fixed a faulty default resulting in borked garbage collection on BOSH deployed workers that are configured to forward through the TSA (they would try to reach the instance IP rather than 127.0.0.1).

fix

Fixed in-place upgrades of binary-deployed workers. Previously registration would fail with a confusing message saying something like "base resource type already exists."

outdated

v4.0.0

July 26, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:4.0.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v4.0.0
garden-runc release: v1.16.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

Who am I?

We've completely redone auth! (Read on before upgrading - this is a huge change and there are some unsupported migration paths.)

In contrast to previous releases of Concourse, users are now central to the authentication flow. Instead of logging in as a team, you now log in as a user and can belong to one or more teams. Users can be added to a team by configuring the team's whitelist as described in Configuring Team Auth.

This is the first step on our march towards full role-based access control. Help us plan that out by checking out the RFC!

In addition, it is now much easier to extend Concourse to support more providers. We're leveraging CoreOS's Dex project for all the moving parts, which already supports a ton of providers (Dex calls them "connectors"). The only delta required for Concourse to support a Dex connector is a tiny bit of glue code in our new Skymarshal component to provide higher-level flags for our CLI.

This was a large change and it was pretty difficult to make backwards-compatible. Here's what's "breaking":

  • There are different flags to pass to the binary distribution, and the BOSH deployment requires manifest changes. Consult Configuring Auth Providers for more information for the binaries, and the Concourse BOSH deployment repo for BOSH.

  • There is no support for configuring the same provider multiple times (say, multiple GitHub Enterprise instances). The migration will fail when trying to upgrade an instance with teams having different configurations for the same provider.

    At the moment, you'll have to deploy multiple Concourse instances. This may be something we can support in the future.

  • There is no longer support for BitBucket auth. Sorry - Dex doesn't support it. :( However we do support generic LDAP, oAuth, and OIDC connectors, which you may be able to use instead.

  • If you have multiple teams configured with the same basic auth username, the migration will fail. This is because "basic auth" is now gone and in its place is local user configuration. Logging in with basic auth is now actually logging in as the configured user, so there can't be multiple.

  • The flags for fly set-team have been split between set-team and concourse web (because part of the config is now global).

  • You may also need to download and install the latest 4.0.0 fly CLI. In the past you would have been able to fly sync your way to the latest version of fly but the new auth in 4.0.0 will cause the old fly to error out.

You should definitely check for these conditions and take a database backup before attempting the upgrade. In practice, our two largest environments upgraded just fine with no intervention required, but if you're not sure, it can't hurt to be careful.

feature

Thanks to Dex, we now also support LDAP based auth!

feature

The fly teams command only lists teams of which you are a member (or all teams, if you're a member of the admin team).

You can also pass -d/--details to show each team's auth config! This should make it a lot easier to check if the auth setup is correct when someone complains about not being able to log in.

feature
breaking

Owing to the auth revamp, the fly set-team command no longer takes flags for the provider configuration (so no client IDs/secrets/etc.). This, in combination with the previous feature, should make tweaking the auth config a lot easier.

feature

The dashboard's influence has taken hold on the web UI! The main page (/) now shows the dashboard instead of some random pipeline configured by the first team on the instance. We've also made the dashboard more powerful with pipeline pausing and re-ordering. We hope you like it because we've removed the sidebar from the pipeline view...it's just cleaner.

We've also spread the colour scheme to the rest of the UI and changed the font everywhere to Inconsolata.

feature

We've made significant improvements to the performance of the build page while keeping its functionality exactly the same.

See this GitHub comment for the nitty-gritty!

There's still more work that could be done, and we put some planning on the issue, but we figured a 6.5x improvement is a good start so we can get back to big juicy features like space. If it's still not fast enough for you, we could really use your help! We're happy to provide guidance for anyone looking to contribute.

fix

Previously, if a resource was only ever used as an explicit output of a job, it would always show up as black even if it was erroring. It will now show up as orange, like the other resources.

fix

We've updated some of the messaging in the UI to be less confusing. When viewing a build that has not been made public it'll now say you're not authorized, rather than telling you to log in, only to tell you to log in again, because that didn't change anything.

feature

When we redid the container lifecycle way back in v3.0.0, one side effect was that containers failed one-off builds would be garbage collected almost immediately, making it pretty difficult to debug (you'd pretty much have to hijack while it was running).

Thanks to a PR from @databus23, there is now a configurable "grace period" after which these containers will be garbage collected! The flag is --gc-one-off-grace-period on the concourse web command, and it defaults to 5 minutes.

fix

We fixed a regression with the CredHub integration that caused very high CPU usage on the ATC. In addition, we've bumped our CredHub client to include a fix PRed by @takeyourhatoff which even further reduces CPU usage. Yay!

feature

The interval on which resource types are checked for new versions can now be set globally via --resource-type-checking-interval, or per-resource-type in a pipeline via check_every.

fix

We fixed a couple situations in the UI where jobs or pipelines with spaces in their name would render incorrectly. (Please don't do this though. It looks so weird and just makes the CLI hard to use! We may have to tighten up naming restrictions in the future, and keeping spaces is pretty low priority. Let us know if you have a real good reason though.)

fix

Clicking the pipeline in the breadcrumb while already on the pipeline page (but viewing a particular group) will now reset the pipeline to the "initial" view. This way it behaves like a normal link.

fix

Repeated team and pipeline creation and destruction would leave a few tables around: team_build_events_XXX and pipeline_build_events_XXX. This would cause the database to increase in CPU usage over time.

We now ensure these tables get cleaned up via database triggers on pipeline/team deletion. However, we decided against writing a migration to automatically clean up existing orphaned tables because, well, it felt scary and dangerous.

If you are seeing symptoms of this problem, it should be safe to manually drop the tables that have no corresponding pipeline or team. We just didn't want to be responsible for a migration that had a high chance of data bloodlust. This way it can be your fault instead of ours!

feature

We've made quite a few optimizations that should take a lot of load off the database. This should improve everything from garbage collection efficiency to web UI response time.

feature

The Vault credential manager backend can now cache credentials based on their lease duration. This was a big chunk of work and should make Vault operators' lives a bit easier. To enable this feature, pass --vault-cache to concourse web. Thanks @rfliam for the PR!

As a side note, we're in need of someone to champion the next phase of credential manager support. We've collected feedback from our first (very much MVP) implementation but really need individuals who have experience with each backend to take the next step. See rfcs #5 for more information!

fix

When running on Windows, we will no longer shell out to tar for performing volume streaming operations, since it seems to be pretty unreliable. A native Go implementation will be used instead, which is a bit slower but much more portable. Thanks for the PR, @ankeesler!

feature

The fly now supports --json on most commands to dump info in JSON format, rather than the human-friendly table format.

fix

Recent versions of Docker introduced an issue where dockerd could fail to start if the worker was under load. This resulted in an infinite loop in the Docker Image resource.

We've made the resource more resilient to this - it'll detect a failure to start and keep resuscitating dockerd until it starts, giving up after 2 minutes.

feature

The S3 resource now supports skip_download: true, thanks to a PR by @talset!

fix

The BOSH release now has properties for configuring the DataDog metrics emitter, thanks to @SHyx0rmZ!

feature

We've split the migration operations out into a separate subcommand: concourse migrate. This is just a bit easier to reason about rather than having all the options baked in to the same command that runs the ATC, and also lets you run migrations without passing all the other flags required by concourse web.

feature

The Prometheus metrics will now automatically prune stale workers, thanks to a PR by @databus23!

feature
breaking

The Prometheus metrics for pipeline scheduling are now counters instead of gauges, thanks to a PR by @databus23!

feature

There are now metrics emitted for peridoc resource checking, thanks to a PR by @databus23!

fix

Fixed handling of no_proxy in concourse worker, thanks to a PR by @databus23!

fix

The Docker Image resource now includes support for fetching and extracting xz packages in ADD commands, thanks to a PR by @et7peho.

feature

The Cf resource now supports no_start: true, thanks to a PR by @klakin-pivotal!

feature

The Docker Image resource now has a tag_file param which deprecates the old tag command which does the same thing. This is in the interest of clearer naming. Thanks for the PR, @ghostsquad!

outdated

v3.14.1

June 7, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.14.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.14.1
garden-runc release: v1.14.0

See the Concourse BOSH Deployment repo for documentation.

There is a known issue where you may see high CPU usage as a result of constant CredHub client connection concourse #2300

fix

Fixed a scheduling performance regression caused by a wonky database index. This upgrade will apply cleanly for those who have manually removed it.

outdated

v3.14.0

June 5, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.14.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.14.0
garden-runc release: v1.14.0

See the Concourse BOSH Deployment repo for documentation.

A migration made it in to this release that slows down scheduling for pipelines with a bunch of version history. v3.14.1 has been released with the fix.

There is a known issue where you may see high CPU usage as a result of constant CredHub client connection concourse #2300

feature

You know that "home" icon that you click all the time and never fully know where it'll take you? Do you remember that empty feeling of not really knowing where "home" is anymore?

WELL IT'S GONE NOW. And you know what's there instead? BREADCRUMBS. Breadcrumbs and memories.

So now you can click the pipeline part to go to the pipeline, or the job part to go to the job.

feature

We've moved the pipeline groups navigation from the top bar to the pipeline page, where it's free to wrap around at its leisure (if there are many groups or looooooooooooooooooooong ones) rather than being constrained to the nav bar.

feature

Container and volume Garbage Collection will now be performed in parallel across the worker cluster.

The ATC is still the source of truth for knowing when containers and volumes are to be removed, but will no longer be responsible for performing the actual "destroy" API calls. This should make large-scale Concourse deployments much more efficient, removing a ton of network and CPU overhead from the ATC.

feature

The Concourse BOSH release now includes Windows jobs: baggageclaim-windows, houdini-windows, and worker-windows. This means you can natively run a Windows worker via BOSH, equivalent in functionality to the Windows binaries!

To see how to deploy it, consult the windows-worker.yml ops file.

feature

The dashboard will now indicate when a pipeline has a resource that is failing to check, by drawing a little orange triangle on the pipeline.

feature

The fly execute command with -j will now use the job's pipeline's resource_types. Huzzah!

feature

fly login can now be invoked with -b to auto-launch a browser to do the oAuth dance, thanks to a PR by @novas0x2a!

feature

The S3 resource now supports configuring an initial version & content, which can be useful for bootstrapping state. Thanks for the PR, @bandesz!

feature

The webhook_token property can now be interpolated using a credential manager, thanks to a PR by @timrchavez!

feature

The Pool resource now supports an atomic metadata update operation, thanks to a PR by @EleanorRigby!

feature

The Git resource now has git-crypt v0.6.0, thanks to a PR by @gcapizzi!

feature

The Prometheus metric emitter has seen some spring cleaning, thanks to @databus23! See atc #274, atc #275, atc #276, and atc #278 for more details.

fix

A couple of the dashboard footer icons looked bloated in Firefox. They're all better now.

fix

We fixed GitHub issue concourse #2000, which is more of a moral victory than anything else. (The fix: the number in the <title> when viewing a one-off build in your browser is now consistent with the number reflected on the page.)

feature

The Git resource now supports two new parameters: submodule_recursive: false, to disable the default recursive fetching, and submodule_remote: true to fetch submodules with --remote. Thanks to @ppaulweber for the PR!

fix

@SHyx0rmZ fixed up a few API endpoints so that they correctly return Content-Type: application/json. Thanks!

fix

Publishing draft releases with the Github Release resource will no longer error, thanks to a PR by @antonu17!

fix

Any errors when checking for a resource's type to have new versions will be bubbled up as resource checking errors. This includes failure to fetch credentials.

feature

The dashboard page now has "Dashboard" in the title.

fix

Fixed an "Aw, snap!" browser crash that affected some versions of Chrome when viewing the pipeline page.

feature

The ATC will no longer fail to start if configured with CredHub and CredHub isn't running. It'll just try and reach it later instead.

fix

The ATC will now fail gracefully early if no session signing key is specified, rather than failing ungracefully and late.

In addition, one will be generated automatically if not given to concourse web. Don't do this forever, though, since users will be logged out whenever you restart the instance, and things won't work at all if you're running a cluster of many web nodes (they all need to have the same session signing key). Thanks for PRing this, @SHyx0rmZ!

fix

We accidentally kept the quickstart command hidden from the concourse --help dialogue. It's there now, thanks to @osis!

feature

The Cf resource now supports client credentials -based auth, thanks to a PR by @jmcarp!

fix

The concourse land-worker command will no longer panic if invoked with no session signing key.

outdated

v3.13.0

April 30, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.13.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.13.0
garden-runc release: v1.13.1

See the Concourse BOSH Deployment repo for documentation.

Jun 1, 2018 A bug in Chrome 67 causes it to crash when loading the Concourse UI. At the time of this notice, the dev/canary versions of Google Chrome should work, as well as other browsers like Firefox and Safari. You can follow along the issue in concourse #2236

fix

Fixed a bug introduced in v3.12.0 concourse #2187 where old build logs were not being reaped. Thanks @aeijdenberg for catching the issue and PR-ing a fix!

feature

Added a new authentication provider for teams using OpenID Connect (OIDC) #2.

Thanks @PavelTishkin!

feature

Concourse can now emit to Datadog using statsd agent #269

Thanks @baptiste-bonnaudet

feature

The semver resource now supports an optional commit_message parameter #64.

Thanks @ElfoLiNk

feature

The dashboard now supports the "not" operator for searches. This can be used on pipeline name searches, team searches, and status searches. Here are some examples:

  • -main gives you every pipeline other than the one called main

  • team:-main gives you every team's pipeline other than the ones belonging to main

  • status:-paused gives you all pipelines that are not paused

outdated

v3.12.0

April 25, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.12.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.12.0
garden-runc release: v1.13.1

See the Concourse BOSH Deployment repo for documentation.

A bug was introduced in build-reaper under ATC/web that causes old build logs to accumulate in Concourse (concourse #2187).

Jun 1, 2018 A bug in Chrome 67 causes it to crash when loading the Concourse UI. At the time of this notice, the dev/canary versions of Google Chrome should work, as well as other browsers like Firefox and Safari. You can follow along the issue in concourse #2236

feature
breaking

The ATC will now batch-delete containers and volumes, rather than making individual calls out to the worker. This is an incremental step towards concourse #1959 that should reduce the network/IO overhead on the ATC during garbage collection.

This requires a new port to be reachable on the worker node from the web node: 7799. This communication will go away once we fully complete parallel GC, as the workers will ask the web nodes what to delete instead.

feature
breaking

In v3.11.0 we changed the default behavior of the Git resource so that it does not fetch tags by default. In hindsight we should have been a bit more careful here, as many people depend on that behavior. We've decided to roll it back to fetching them by default, since in most cases (where tags are never deleted or re-pointed) it's not an issue to include them.

To disable fetching of tags, configure clean_tags: true in params.

Thanks for the PR, @mdomke!

feature

The ATC can now be configured with a global default build_logs_to_retain, thanks to a PR by @aeijdenberg! This is useful for operators who want more control over their database usage.

A maximum value can also be configured to ensure users don't just set it to 9 trillion. The flags are --default-build-logs-to-retain and --max-build-logs-to-retain.

fix

Fixed a memory leak in the TSA introduced in v3.11.0.

feature

The fly set-pipeline command will no longer prompt apply configuration? if there are no changes to apply.

feature

There's a fly order-pipelines command now, for setting the order of pipelines.

feature

A fly status command has been added for checking whether or not you're logged in to the given target.

feature

When fly check-resource fails, it'll bubble up the error message rather than just saying error code 500.

fix

Work around an apparent regression/behavior change in recent versions of Chrome that prevented the pipeline UI jobs from being clickable.

fix

Fixed a corner case in error handling that could cause a lock to be held forever when detecting new versions of resource types. This could lead to things like builds stuck in "pending" state. Thanks to @SHyx0rmZ for the PR!

fix

When directed to the login page from the resource page, you will now be redirected back to where you were, rather than to the moon.

fix

The concourse web command is now capable of running the migration flags (--current-db-version and friends). It's still super janky (it'll run the TSA alongside your...migrations), but hey, it runs them now.

We'll probably clean this up in the future by having the migration stuff be a separate command instead.

fix

fly will once again helpfully instruct you to log in rather than just saying error: forbidden.

feature

The BOSH release now supports properties for configuring the new AWS Secrets Manager credential backend. Thanks for the PR, @x6j8x!

feature

When a previously-created volume disappears from a worker and the ATC tries to use it, the error message will now include the worker name and the volume handle. Thanks for the PR, @rfliam!

fix

The fly check-resource command will now fail more clearly when the resource's type is not found. Thanks to @jmcarp for the PR!

outdated

v3.11.0

April 13, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.11.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.11.0
garden-runc release: v1.12.1

See the Concourse BOSH Deployment repo for documentation.

v3.11.0 has a memory leak issue in the ATC/web server concourse #2165. The severity of the memory leak will be different based on the size of your deployment and the types of workloads you run on it.

fix
breaking

The Git resource will no longer have tags present in the fetched repo, thanks to a PR by @benmoss.

They are now cleared out after the fetch, because the state of all tags is prone to change after the initial fetch, as the resource's source of truth is commits. So after the fetch, the cache would have an out-of-date view of the tags, which could lead to problems when pushing.

fix

Fixed a bug in the BOSH release that prevented the worker nodes from starting in a fresh deploy. Sorry about that. Thanks @z4ce for the PR!

feature

The /dashboard page looks better on phones now.

feature

The /dashboard page makes way fewer requests now, so it's a lot faster to load and more efficient to periodically refresh.

feature

The fly builds command can now filter by team (-t) or pipeline (-p), thanks to a PR by @andrewedstrom!

fix

Fixed a couple migrations that assumed a public schema. Thanks to @vganoradg for the PR!

fix

fly will no longer repeatedly detonate when given an invalid token during fly login.

feature

There's a new credential manager in town for AWS's newly-launched Secrets Manager (not to be confused with Systems Manager, which is also used for managing secrets). Thanks @x6j8x for the PR!

We realize that we now have two undocumented credential managers, one called AWS SSM and one called AWS SM. We like to call this "hard mode". (Sorry, we'll backfill docs soon.)

feature

The Using Vault credential manager can now be configured with a --vault-auth-backend-max-ttl, after which it will force a re-login. Thanks @baptiste-bonnaudet for the PR!

feature

The Using Vault credential manager will now retry with exponential backoff when logging in, rather than retrying every second.

fix

The Time resource will now correctly handle a tricky configurations that span multiple days (e.g. 10AM - 5AM), thanks to a PR by @jleben!

feature

The Git resource will now make the commit message accessible under .git/commit_message, thanks to a PR by @elgohr!

feature

The web node can now be configured with --cookie-secure to force secure: true on its cookies. Thanks for the PRs, @jmcarp!

feature

The Github Release resource now supports a tag_filter configuration for matching arbitrary semver tags, thanks to a PR from @jmcarp!

fix

Added a missing property to the BOSH release for configuring the Generic oAuth provider's CA cert. Thanks for the PR, @youngm!

feature

The Docker Image resource now supports configuring aws_session_token, thanks to a PR by @itsdalmo!

feature

The Docker Image resource now has yet another new param, cache_from, thanks to a PR by @dhinus!

This new param is like load_bases except everything loaded will also be used as a cache during the build.

fix

The Git resource will now recover from a deleted tag when configured with tag_filter, thanks to a PR by @ljfranklin!

feature

fly validate-pipeline with --strict will now be more strict with your YAML, thanks to a PR by @aeijdenberg!

feature

The Cf resource now supports verbose: true, which will tell the CLI to dump trace logs to the output. Thanks for the PR, @JohannesRudolph!

feature

The Docker Image resource now supports a target_name param for specifying the target to build in a multi-stage Dockerfile. Thanks to @irfanhabib for the PR!

feature

The BOSH release now bakes in the glue code for use with BOSH Backup and Restore, thanks to a PR by @rkoster!

feature

The fly set-pipeline command can now be given --no-color flag to strip out the color from the diffs. Instead of using color, + and - will be at the start of added and removed lines.

feature

Now that we're building with Go 1.10+, fly will respect socks5 proxies configured via the "standard" http_proxy/https_proxy env vars.

outdated

v3.10.0

March 29, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.10.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.10.0
garden-runc release: v1.12.1

See the Concourse BOSH Deployment repo for documentation.

Users upgrading to v3.10.0 with BOSH have reported issues where baggageclaim will not start after the upgrade. This issue has been reported and fixed for future versions of Concourse. In the meantime, you can workaround the problem by using the workaround described in #2125

breaking

This release involves a worker protocol version bump, from 1.3 to 2.0, and also switches the default BaggageClaim driver back to btrfs. Read on for more information!

We recommend spinning up a new pool of worker nodes, upgrading your web nodes, and then removing the old workers. Otherwise your workers may get swarmed with containers, if only one 2.0 worker is added at a time with the web nodes already upgraded.

feature
breaking

The concourse worker commands can now be pointed at multiple TSA addresses, rather than one, so that it can retry against a random node each time. As part of this, we've removed the --tsa-port flag and changed --tsa-host to instead take a host:port. (Because TECHNICALLY, they could be on different ports.)

feature

We've revamped how fly execute gets its inputs and outputs to/from the build, so that configuring the ATC with an external URL is no longer required. See concourse #2069 for the nitty-gritty.

feature
breaking

We've switched back to btrfs as the default driver. We switched away in v3.1.0 in the midst of a ton of stability issues, which we have think we've fixed in v3.9.0.

This resolves a long-standing performance issue when using privileged tasks or resource types (like the Docker Image resource). For more information, see concourse #1404 and concourse #1966.

Be sure to use the latest possible kernel version so that you have a btrfs with the latest fixes. We suspect that this will still be an occasional issue, though far less frequent.

fix

Previously, tags on a resource type didn't get no respect. They are now respected.

fix

Fixed an ATC crash that would occur when a task step errored with the next step using an attempts step modifier.

feature

The concourse binary (and Docker image) now supports a quickstart command, which will spin up an itty bitty Concourse cluster with a single worker.

feature

The Docker Image resource now supports pushing multiple tags, thanks to a combined effort of @gcmalloc and @jerith!

feature

When the ATC is streaming data between workers, the stream will now be gzip-compressed, which should speed things up quite a bit. (This is what caused the worker version bump to 2.0.)

feature
security

The ATC now requires TLS v1.2+ and a stricter set of cipher suites. Say that five times fast!

fix

Concourse now supports the newer umask-hardened BOSH stemcells (v3541.x).

fix

Fixed a botched bashism that led to the Docker Image resource to exit early on certain environments (more info here).

fix

Cleaned up a noisy PostgreSQL error that would occur on start when checking for the legacy migration_version table.

fix

Fixed a UI glitch that caused the last line to be misaligned with the timestamps if it had no trailing linebreak.

outdated

v3.9.2

March 12, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.9.2

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.9.2
garden-runc release: v1.12.0

See the Concourse BOSH Deployment repo for documentation.

feature

Due to popular demand, we're graduating the pipeline dashboard out of beta and back into /dashboard. You can now also do useful things like log in and click through to the normal pipeline and build pages.

feature

The NewRelic metrics emitter has now been improved thanks to @novas0x2a!

fix

Removed a pesky database constraint (cannot_invalidate_during_initialization). This would occasionally bubble up to the user in weird ways, and actually isn't necessary any more.

fix

The previous release snuck some code that wasn't quite ready yet into the BOSH release and broke registration of external workers. This is now fixed.

fix

Occasionally builds would fail when interacting with Vault with http2: no cached connection was available. NO LONGER! (We, uh, we bumped a dependency.)

fix

Certain ANSI cursor movement escape sequences would wreak havoc on the Concourse build output page because there was no window size set on the TTY, thus defaulting it to 80x24. We've set it to 500x500. That oughtta do it.

fix

Fixed an issue where Firefox users couldn't click around on the build page.

outdated

v3.9.1

February 26, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.9.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.9.1
garden-runc release: v1.11.1

See the Concourse BOSH Deployment repo for documentation.

feature

The build page will now render exotic ANSI text modes like faint text, framed text (...?), and... Fraktur (??????).

This is a feature, but I'll be damned if it's what makes us go to 3.10.

If you're wondering what Fraktur looks like, check out the pull request. Thanks, @evanphx!

If you're wondering why that's a standard, keep wondering.

fix

We've optimized the rendering of the build page, which got quite a bit slower with the introduction of timestamps in v3.6.0. There is more work to do here, but we've added performance tests to catch any future egregious regressions.

fix

The BOSH release's credhub.client_id and credhub.client_secret properties are now respected once again. Ironically this broke when we added CredHub testing to our pipeline, but in a way that avoided UAA client ID/secret configuration as it was much too heavyweight (hence the introduction of TLS auth properties).

fix

Fixed a BOSH release template resolution error from an undefined esc method, which would only occur on a clean deploy thanks to the magic of global state. This is an example of why using Ruby to generate Bash scripts is a terrible idea.

Thanks to @calebwashburn for discovering the issue and PR-ing a fix!

fix

Cleaned up those pesky atc.skymarshal.user.not-authorized logs.

fix

Fixed a crash that would occur when a task step configured image but no config or file.

fix

The fly CLI will now buffer output when rendering tables, which should make things a bit faster on Windows.

outdated

v3.9.0

February 13, 2018
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.9.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.9.0
garden-runc release: v1.11.1

See the Concourse BOSH Deployment repo for documentation.

  • There is a known issue with the BOSH release of v3.9.0 where the ATC will fail because function esc is not defined concourse #2029

  • CredHub integration has a bug in this release, please see concourse #2034 for more details

feature

Certificates can now be automatically propagated from the worker machine into resource containers. This feature took a lot of thought, trial and error, and shaking our fists at each Linux distribution maintainer for having a different approach to how certificates are stored and managed. There's a lot of context for this in concourse #1027.

Long story short, we wanted a way for certificate management to be done generically across all resources, so that resource authors don't have to keep implementing various forms of ca_certs, insecure_skip_verify, etc., and things can "just work" securely by default.

This feature is enabled by default for our BOSH distribution. The concourse binary now has a --certs-dir flag on the worker command, which should be given something like /etc/ssl/certs or /etc/pki/tls/certs depending on your Linux distribution.

See Certificate Propagation for more information.

fix

The btrfs volume driver should now be much more stable. This resolves a (very) long-standing issue that caused us to switch the default to overlay, which in turn introduced a known performance regression with privileged tasks and resources (i.e. Docker image building).

If you're feeling the pain of overlay, we recommend switching the driver to btrfs now and letting us know if you see any issues. Initial feedback is positive. If things are looking good we may switch the default back to btrfs.

feature

Tasks now support inputs.optional inputs, thanks to a series of PRs by @rosenhouse! This is great for adding optional behavior to tasks and incrementally changing them backwards-compatibly.

feature

The CredHub credential management integration can now be configured with mutual TLS based authentication.

feature

Teams can now be renamed via rename-team. (...yay!)

fix

The Docker Image resource now correctly handles complicated build args, thanks to a PR by @jfmyers9 and @ljfranklin.

feature

The target will no longer be deleted when running fly logout - only its token.

feature

When viewing a resource, it will now show when it last checked.

fix

The S3 resource will now auto-adjust the part size when uploading, so that it can upload files over 50GB. Thanks @ruurdk for the PR!

feature

The Docker Image resource now supports loading multiple images at the start via load_bases for use in multi-part Dockerfiles, thanks to a PR by @krishicks!

fix

Multi-part Dockerfiles with multiple ECR images will now correctly pull each with ECR login support, thanks to a PR by @PeteGoo.

feature

When using fly intercept with --url, the appropriate target will now be auto-detected based on the URL. Thanks @jmcarp for the PR!

fix

Reduced the throttling when talking to k8s for credential management, thanks to a PR by @william-tran.

feature

The Prometheus metrics endpoint now includes scheduling and database metrics, thanks to a PR by @TimSimmons!

fix

The Prometheus metrics endpoint no longer breaks HTTP metrics down by path, because that made the cardinality too damn high. Thanks again @TimSimmons!

feature

A NewRelic Insights metric emitter has been added, thanks to a PR by @novas0x2a!

feature

Support for using AWS SSM for credential management has been added, thanks to a PR by @surajbarkale!

feature

When viewing a pipeline on a teeny tiny display, the obnoxious color indicator legend thingy will no longer prevent you from clicking the stuff below it. Thanks @SwamWithTurtles for the PR!

feature

The Cf resource now has a show_app_log config for tailing the app logs while starting it up. Thanks for the PR, @aeijdenberg!

feature

The Docker Image resource will now propagate http_proxy and https_proxy when building docker images, thanks to a PR by @boazy!

feature

The Docker Image resource can now be configured with max_concurrent_downloads and max_concurrent_uploads, thanks to a PR by @drahnr!

feature

The Github Release resource will now produce a commit_sha file containing the...commit sha that the release's tag points to. Thanks @defsprite for the PR!

fix

When contacting CredHub, the configured CA cert is now respected. It was ignored in previous releases. Sorry about that! We've ramped up testing in our pipeline to catch silly things like this in the future.

fix

Fixed finicky 500 errors when running fly volumes as a result of volumes disappearing while the API walks through and gets their info. The endpoint is still slow, but it at least won't blow up in this case.

fix

Fixed missing validation for on_success, on_failure, and ensure when configured on a job, thanks to a PR by @jmcarp!

fix

Fixed a subtle timing issue that could result in fly watch not finding any builds to watch when given a job.

outdated

v3.8.0

December 6, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.8.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.8.0
garden-runc release: v1.9.0

See the Concourse BOSH Deployment repo for documentation.

Y'all really carried us on this release. We just shipped it. Thanks everyone!

feature

The ATC can now be configured with an idle timeout for fly intercept sessions, thanks to a PR by @sharms and @jmcarp!

fix

In v3.7.0 we broke the ability for BOSH deployed workers to configure an external TSA - the known_hosts file would be empty on the machine. Sorry about that! This should work again. Thanks for the PR, @christianang!

fix

Also in v3.7.0 we mistakenly provided a default empty value for the now-required token_signing_key property on the atc and tsa jobs in the BOSH release. This is now removed, so you can get a nice juicy template resolution error rather than a busted deployment.

feature

The container placement strategy can now be configured in the BOSH release, thanks to a PR by @williammartin.

feature

The Generic oAuth provider can now be configured with a CA certificate, thanks to a PR by @rkoster!

fix

Fixed a typo in the new Prometheus metrics endpoint that prevented volume count metrics from being emitted. Thanks @jmcarp!

outdated

v3.7.0

November 30, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.7.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.7.0
garden-runc release: v1.9.0

See the Concourse BOSH Deployment repo for documentation.

This release has issues. Primarily with the BOSH distribution. You should probably just skip straight to v3.8.0.

feature
security
breaking

We've ripped out the old & janky PostgreSQL job from our Concourse BOSH release. You will have no choice but to bring your own PostgreSQL database.

If you use the Concourse BOSH release and you haven't upgraded in a while, I'd suggest you check out the previous migration instructions from Concourse v3.5.0 and v3.6.0.

feature
breaking

We've changed how we develop database migrations, so as to support down migrations in the future. This will hopefully mean that if you upgrade Concourse and for whatever reason need to back out, you'll be able to, instead of being stuck on a (possibly broken) latest version.

As part of this switch, we've also squashed our migrations into one big bang, which should also improve startup time for fresh installations. However, this means that you must first upgrade to v3.6.0 before upgrading to v3.7.0!

So, do that. You may need to anyway now that we've removed PostgreSQL (see previous note).

feature
breaking

There are new required manifest changes to deploy this release with BOSH.

Our BOSH release used to have a few magical mystical packages called generated_something. These packages would generate a RSA key every time they compiled, in service of automagically wiring up security credentials so you didn't have to put them in your manifest.

This approach was extremely clever and whoever came up with the idea was a downright genius, way ahead of their time.

We've now collectively decided that the whole approach is stupid and redundant now that BOSH manifests can generate their own typed variables. It was fun while it lasted.

What you need to do for this change is described in concourse #1834. You can consult our changes to manifests/single-vm.yml for reference.

As an alternative to hand-editing your manifest, the next release note may pique your interest.

feature

We have started dusting off concourse-deployment and using it as a central location for Concourse BOSHy deployment goodness. We now use it for our production deployment, as well as a few testing environments.

feature

We are now openly gathering feedback on one of our worst-kept-secrets: the Concourse dashboard view. You can access it by visiting /beta/dashboard.

In this version of Concourse, we've tweaked some of the visual elements of the dashboard to make it more readible for installations with multiple teams. We've also fixed some of the pipeline states so that they make more sense.

Tell us what you think about the new dashboard by dropping us a line on concourse #1829.

feature

The fly execute command will now default to -x, which has been replaced with a new flag, --include-ignored, to revert to the old behavior.

In addition, Fly will no longer blow up when trying to execute with an input that doesn't have a .gitignore. It will also not blow up if any inputs are files and not directories.

feature

The ATC will now use a separate database connection pool for the API and the pipeline scheduling work. This will make it so that a bunch of slow API requests can't starve critical functionality.

feature

Pipeline-provided resource types will no longer fail miserably for a minute or two when they're first configured.

feature

You can now configure params on a pipeline resource type, thanks to a PR by @jghiloni! This will enable users to use the S3 resource with params: {unpack: true}, as an alternative to the Docker Image resource.

feature

You can now specify a on_abort hook on a step or on a job. It will run on abort. 🎉

feature

The ATC can now be configured with a pure-random worker selection strategy, which may help users affected by our default resource affinity placement, which can result in overloaded workers. This is thanks to a PR by @phillbaker!

To use the random placement strategy, pass --container-placement-strategy=random to the web command.

feature

The fly jobs command now has a column indicating whether any builds are pending or started for each job, thanks to a PR by @rowanjacobs!

feature

The S3 resource now supports being configured with a session token, thanks to a PR by @keymon!

feature

Git repos encrypted with git-crypt will now be automatically decrypted by the Git resource, thanks to a PR by @dmrschmidt!

feature

The ATC can now be configured to serve a metrics endpoint for Prometheus, thanks to PRs by @TimSimmons and @jmcarp!

feature

Teams can now have BitBucket-based auth, thanks to a PR by @SHyx0rmZ!

feature

The Git resource can now be configured with a HTTPS proxy, thanks to a PR by @jghiloni!

feature

Inline task configs are now validated as part of pipeline validation, thanks to a PR by @jmcarp!

feature

The Cf resource can now be configured with a Docker username/password for pushing an app using a private repository, thanks to a PR by @elgohr.

feature

The Github Release resource now supports being configured with insecure: true to support private GitHub Enterprise installations. For the long-term strategy regarding this, see concourse #1027.

feature

The Semver resource now supports being configured with skip_ssl_verification: true to support private S3-compatible blobstores, thanks to a PR by @calebwashburn.

Notice how this note and the prior note have entirely different property names for doing the same thing. Blargh! See concourse #1027.

feature

ATC now has a flag for using k8s secrets when running in a cluster. This change makes using the k8s credential manager an explicit choice when running inside k8s, and also allows you to use a different credential manager when running in a cluster. Thanks for the PR and the patience by @william-tran and @farcaller

fix

When the ATC is configured with multiple metrics emitters, it will now error, rather than silently picking one, thanks to a PR by @jmcarp.

fix

Fixed an issue where selecting/copying the build output would also select the timestamp on the left.

fix

fly login will now error if arguments are mistakenly given to it.

fix

Turns out you could easily spam the build page by holding T to trigger multiple builds. We've fixed that now so it only triggers a build once. #YOTO

fix

Fixed the web UI so that it appropriately shows that you are logged out when your session expires.

fix

The deprecated Bosh Deployment resource resource now, uh, contains the bosh CLI again. Sorry about that. Switch to the CloudFoundry BOSH deployment resource if you can though!

This was fixed by a PR by @Infra-Red. Thanks!

fix

Fixed an issue with the CredHub integration that made it necessary to configure --insecure-skip-verify, thanks to a PR by @aeijdenberg!

outdated

v3.6.0

October 25, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.6.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.6.0
garden-runc release: v1.9.0

See the Concourse BOSH Deployment repo for documentation.

feature
security
breaking

Concourse now requires PostgreSQL v9.5+. If you're already up-to-date, you've got nothing to worry about.

If you're using the BOSH release, and have already followed the instructions in v3.5.0, you're all set.

If you haven't, do so! You should be able to do the same upgrade process with this release, except that some queries will be failing in the ATC until everything is upgraded.

With PostgreSQL v9.5.0+ we've been able to dramatically improve performance of the pipeline UI. It's worth it!

feature
breaking

We've bumped our Garden-runC dependency to v1.9.0. This upgrade requires a recreate of your workers.

feature
breaking

The fly validate-pipeline will now validate the config field on embedded tasks.

As part of this change, we have removed support for configuring both config and file, which was deprecated about a billion years ago and has been emitting shouty warnings all this time.

feature

Build logs now have timestamps! In addition, you can click them to link to lines, and shift-click to select ranges! WHEE!

feature

There are some fancy-schmancy keyboard shortcuts on the build page. Press ? on the build page to learn more!

fix

Fixed a case where a bunch of pipeline scheduling happening at once could result in a client-side database connection limit being reached, resulting in slowness.

fix

Previously, when clicking and dragging in the pipeline UI, if the initial click was on a job, it would take you to the job when you let go. NO LONGER!

outdated

v3.5.0

September 19, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.5.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.5.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature
security

Those of you using our BOSH release have been stuck with either our way-out-of-date-and-vulnerable-to-thousands-of-CVEs-and-EOL-next-year version of Postgres (9.3), or an external Postgres server. NO LONGER!

We've enabled an upgrade path to the CloudFoundry Postgres BOSH release, which is up-to-date (currently 9.6.4) and even supports release-to-release upgrade paths!

The next version of Concourse will require version 9.5+ of Postgres, so action is required either now or upon the next release to upgrade.

The postgresql job in our release should now be considered deprecated, and will be removed in the next release.

To switch off from our pitiful pre-packaged precarious Postgres, do the following:

  • First, deploy this version of Concourse with no changes. We have modified our Postgres job to move its data to a new location that the Cloud Foundry Postgres release will upgrade from.

  • After the deploy has finished, upload the Cloud Foundry Postgres release. We've tested the upgrade path with version 22, available on bosh.io.

  • Once the release is uploaded, add it to your deployment manifest, swap out the concourse/postgresql job for the postgres/postgres job, and update the ATC properties to explicitly configure the database and role. (That's a mouthful, but you can use our changes to the single-vm manifest as a reference.)

  • Note that the Postgres DB upgrade must not be combined in the same deployment operation with a stemcell update.

feature

Concourse now supports CredHub for external credential management. See Using Credhub for more information.

fix

GC no longer creates crazy seesaw patterns with containers and volumes. This was an issue that was introduced in v3.4.1 when 2 or more teams have identically configured resources. It's been fixed now.

fix

Fixed a leak with goroutines that happens from fly intercept

feature

When using groups in pipelines, fly will now let you know when you forgot to assign a job to a group. Say goodbye to hidden pipelines!

fix

Jobs and resources with a forward slash in their name no longer error out when loading their details.

feature

The Semver resource now supports Google Cloud Storage. Thanks @chendrix!

fix

The S3 resource correctly shows the progress of uploading and downloading. It will no longer report 2 TB/s. Thanks @krishicks!

outdated

v3.4.1

August 28, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.4.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.4.1
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

breaking

Looks like the GC "fixes" had the opposite effect of making containers linger around longer than usual. You might want to skip this version for now. Otherwise, hang tight while we get this sorted out.

breaking

Removed a feature introduced in v3.4.0 that lets you authenticate into a team using personal GitHub tokens.

GitHub's API surprisingly accepts oAuth tokens in the same flow as access tokens. This makes the GitHub auth flow supported by Concourse less secure, as oAuth tokens may be acquired by an exploited third-party service that users have authorized, thereby allowing them to log in to the user's CI system.

feature

The fly command for fly set-team and fly destroy-team now lets you supply the flag --non-interactive. Such automation. Amaze.

Thanks to @aleksey-hariton for the PR

feature

BaggageClaim volume creation APIs are now asynchronous; this should remove the need for crazy timeouts.

Thanks to @SHyx0rmZ for implementing this!

feature

fly now prints a URL to your build page when you run fly execute. How convenient!

fix

Deleting teams would cause the garbage collector to freak out and cause a buildup of worker containers.

That's been fixed now.

fix

Added the appropriate headers to stop GitHub from caching badges.

Thanks to @belljustin and @cunnie for fixing this longstanding issue!

fix

Fixed an issue where the pipeline view will reset after a state change on the pipeline.

fix

Previously, if a resource or resource type was parameterized via a credential manager, its check containers and caches would be mistakenly garbage-collected. They will now be kept around.

fix

Check containers will no longer be brutally destroyed if they're used too close to their expiration time.

outdated

v3.4.0

August 15, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.4.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.4.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature

We've deprecated our concourse/lite Vagrant box in favor of a bosh create-env flow. This will be much easier for us to maintain and brings a lot more flexibility around configuring and upgrading Concourse.

feature

We've parallelized garbage collection. This should make things more durable to a slow worker, and make it harder for containers and volumes to "pile up" when the ATC is out of service briefly (i.e. during a deploy). Yee.

feature

The legend on the pipeline page will now auto-hide after 10 seconds.

feature

When switiching between pipelines, the UI will now fit the pipeline in view.

feature

You can also press 'F' to pay respects center a pipeline in view.

feature

You can now log in with a personal access token when logging into a team with github auth, thanks to a PR by @Chumper!

feature

You can now set image_resource.version on an image_resource, thanks to a PR by @krishicks!

fix

We've removed the volume size column from fly ... previously it was always empty and no one seemed to care.

fix

The fly validate-pipeline command can now be provided with variables in the same way that fly set-pipeline can, thanks to a PR by @jmcarp!

feature

The Bosh Deployment resource now uses the latest BOSH CLI, thanks to a PR by @selzoc!

feature

The Semver resource now supports Server Side Encryption, thanks to a PR by @miromode!

feature

The Git resource will now save the committer email to `.git/committer`, thanks to a PR by @knifhen!

feature

Jobs with a pending build now have a static halo to better represent its waiting state, thanks to a PR by @d!

feature

fly learned the fly format-pipeline command, thanks to a PR by @krishicks!

feature

The fly abort-build command can now abort by build ID, thanks to a PR by @kurtmc!

feature

BaggageClaim's response header timeout is now configurable, which should help those with large images that they're using for privileged tasks. This is a band-aid; we'll soon be making the API this is relevant to async.

fix

Files with the setuid and setgid permissions set on them will no longer have them removed. This used to be lost with the chown performed for namespacing the files. We'll now restore it after the chown.

fix

The flags for configuring GitLab oAuth are now present in fly set-team.

fix

Fixed an underflow in BaggageClaim's volume size detection, thanks to a PR by @SHyx0rmZ! This affected deployments with less than 10GB of space. (Psst: you should probably get more anyway.)

outdated

v3.3.4

August 1, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.3.4

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.3.4
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

A migration introduced in v3.3.0 would load all the builds into memory and then process them, causing a lot of issues when upgrading. We optimized this migration to migrate build plans in batches, rather than all at once.

fix

The unpack support for the S3 resource will no longer load the entire archive into memory, so it can be used for larger archives, thanks to a PR by @krishicks!

outdated

v3.3.3

July 24, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.3.3

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.3.3
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature

Added support for params in image_resource. This probably should've always been there, but hey at least its there now!

feature

The S3 resource supports unpack, which unarchives a tar/zip file on the get command. This enables s3 to be a provider for image_resource docker images. Thanks to a PR by @krishicks!

outdated

v3.3.2

July 11, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.3.2

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.3.2
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix
security

Fixed a vulnerability affecting installations using untrusted multi-tenancy (i.e. multiple teams who may be jerks). This issue affects all versions after and including v2.7.1.

If you are running a single-tenant Concourse installation, or an installation where all team members are "trusted" (i.e. part of your small org), you don't have much to worry about. Otherwise, you'll want to upgrade to this as soon as possible.

We strongly recommend upgrading as soon as possible.

feature

The fly execute command with -j can now resolve the Vault credentials configured in the job's inputs.

fix

ECR support is now fixed in the Docker Image resource.

outdated

v3.3.1

June 30, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.3.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.3.1
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature

The fly volumes command will now show task cache volumes as task-cache and show the name of the task the cache is for.

fix

The last release (v3.3.0) broke resource configs with nested maps in them. This is now fixed.

fix

Fixed an ATC crash that would occur when trying to list pipelines while the database is down. Along the way we also made the endpoint quicker.

outdated

v3.3.0

June 28, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.3.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.3.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

breaking

A migration introduced in this release loads all the builds into memory and then processes them. It has to do this (rather than read one row at a time and update) as it only has one transaction. This might cause some memory issues on the ATC while upgrading. This migration has been fixed in v3.3.4

feature

Our first pass at support for credential management has landed! With this you can externalize all of your credentials in to Vault (more providers coming later), preventing your credentials from ever entering the database and allowing for automatic credential rotation.

For more information on how to configure and use this feature, see Credential Management.

feature

Tasks now support caching arbitrary paths by configuring caches in the task config. This can be used to speed up builds that fetch dependencies at runtime or compile into a common directory (e.g. pkg for Go). For more information, see caches.

feature
security

In our initial support for encryption, we missed a spot. Build plans (an internal structure generated when a build starts) were previously saved into the database in plaintext, and would sit around forever.

Now, we encrypt them and remove the build plan (encrypted or not) when the build completes.

fix

Previously, if a serial group had a paused job in it, and the job had a build queued up, the entire serial group would wait for this build that would never run, forcing you to continuously abort the pending builds to unwedge your pipeline.

It...doesn't do that now, thanks to a PR by @jmcarp!

fix

A previous release broke rendering of older builds. They'll now render properly.

feature

The web UI will now serve back a 404 page when the content you requested is not found, rather than just...being broken.

feature

The login prompt will now tell you if your basic auth credentials were invalid, rather than leaving you to sit and think about what you've done wrong.

feature

Concourse now supports GitLab oAuth configuration, thanks to a PR by @markstgodard!

outdated

v3.2.1

June 16, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.2.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.2.1
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

countdown

feature

The ATC now garbage-collects containers and volumes in parallel, rather than garbage collecting all the containers and then all the volumes. We'll make this even more parallel in the future (while being careful not to swarm the workers).

fix

A migration introduced in v3.2.0 to ensure there's only one volume per resource cache on a worker would fail if there were already duplicates, making the ATC unable to start. The migration will now nullify the duplicates so that they become garbage-collected.

fix

We accidentally broke handling of strings passed from -v to {{params}} by making them actually be parsed as YAML values, for ((params)). Turns out that was confusing even for ((params)), so we've made it so that -v is always for verbatim strings, just as before, and added -y for the less-frequent case of wanting to provide a YAML value as a parameter.

fix

The ATC became very picky in the last release around extra keys in your pipeline config. It will now permit extra toplevel keys, and only raise a validation error for extra keys nested under jobs, resources, etc., so you can continue to declare values at the toplevel for YAML anchoring.

outdated

v3.2.0

June 15, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.2.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.2.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature

We've pulled in a newer, less crappy templating system for fly! New params look like ((this)) and support more than just strings: boolean values, arrays, and other YAML structures can be templated in. It also supports one big thing you've all been waiting for: inter((pola))tion!

The older, rougher-around-the-edges {{og-params}} are still supported and behave exactly as they did before. You should switch to the new style at your earliest convenience, but we have no immediate plans to remove the old style as it's really not that annoying to just leave in place.

fix
breaking

As part of our params rejigger, we no longer support specifying maps as arrays of maps. We have no idea how or why this worked previously, but we never intentionally supported it, and won't be bringing this back. For slightly more context, see concourse #1307.

feature

The Docker repository now supports resolving addresses within Concourse containers via Docker's magic local DNS server! This fixes a longstanding hurdle our Docker users would run in to pretty frequently when wanting to e.g. point Concourse at other Docker-deployed things like a registry within the cluster. It also removes the need for setting CONCOURSE_GARDEN_DNS_SERVER.

No configuration is necessary for this change.

feature

We've made substantial improvements to our schema that should dramatically reduce utilization of your database, especially for larger deployments. On our own instance we saw Postgres CPU usage drop from ~25% to ~7%. Larger instances will likely see a more substantial impact.

feature

fly now supports tab-completion of pipeline names, thanks to a PR by @jmcarp!

fix

As part of the schema improvements, we've fixed a couple edge cases that could result in a container or volume for a get step being brutally murdered in the middle of a build running. We will now also wait for get steps to finish when draining a worker.

We're sorry, get step. You deserved better.

fix

We've fixed a goroutine leak and generally redone how the radar component of the ATC scans for resource versions. As a result of this, the goroutine count dropped by just over 50%.

outdated

v3.1.1

May 31, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.1.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.1.1
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

Fixed a race condition during volume creation for worker base resource types. This would result in resource actions failing with failed to create volume.

Unfortunately, to propagate this fix you'll need to recreate your workers, as they're in a broken state.

fix

Fixed the reporting of database query metrics.

outdated

v3.1.0

May 29, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.1.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.1.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

Resources are now unprivileged by default. We've been wanting to do this for a while for security reasons (especially in the context of custom resource types), but we ended up needing to do it anyway as part of the change to overlay (two release notes down), in kind of a roundabout way.

To make a custom resource type privileged, configure privileged: true.

This change should only require action from folks using a custom resource type that actually has depended on being privileged, such as any forks of the Docker Image resource. It may be that you (or the resource author) will not know this until it fails. Resource authors are encouraged to update their READMEs if their resource type does indeed need it.

This was motivated by the one downside of overlay that we encountered: namespacing a volume takes much longer. Namespacing a volume means re-mapping its files to have UIDs that correspond to the correct UIDs for its target container's user namespace. This typically a recursive chown that remaps host-side UID 0 to the container's equivalent (UID 4 billion something), or vice-versa.

Namespacing a volume takes longer for overlay because a chown of a file necessitates copying the file to the upper layer, which takes time and space. This did not affect btrfs as the nature of its copy-on-write mechanism is vastly different.

Most workloads should be unprivileged, so to optimize for that, we've made all resources unprivileged by default. This means that they generate unprivileged volumes which won't need to be namespaced for unprivileged tasks (and other resources, i.e. put).

feature

Pipeline config and team auth settings can now be encrypted in the database. See Encryption for more information.

This is the first step towards a better security model around credential management. The next step will be to externalize credentials entirely to a credential manager such as Vault or CredHub.

feature

Workers will now use overlay rather than btrfs as their filesystem of choice for BaggageClaim.

If you're morbidly curious about all the low-level mumbo jumbo that went into this venture, check out issue concourse #1045.

If you don't really care, know that upon upgrade the workers will stick with btrfs, for backwards-compatibility. It is only on recreate that they will pick overlay, and only if it makes sense to (i.e. your kernel version is > 4.0, and your disk is not formatted as btrfs already).

There are many benefits to what sounds like a low-level change:

  • No more loopback shenanigans. Most of you aren't using btrfs as your disk's filesystem, so Concourse would always have to make a sparse file and mount a loopback device over it. This was just confusing and made disk usage hard to interpret, and possibly leak.

  • No more locked-up workers! Many users, including us, encountered a btrfs bug which led to reads and writes of the disk locking up. The only way out from this was to recreate the worker.

  • Concourse in Docker for Mac should now work! They had btrfs stripped out of the kernel image, so Concourse had to fall back on the naive driver, which is hella slow.

  • A noticeable performance boost under load. In our testing, overlay performs better under realstic work loads like building many Docker images at once. While overlay would edge out btrfs at low contention, it performed even better as we ramped up the concurrency. What took btrfs 10 minutes would take overlay around 7.

    Also worth noting is that during testing we could pretty reliably get btrfs to lock up. It actually made it hard to collect enough data. But maybe that's a more important point than any set of numbers.

fix

Fixed metadata for fetched resources often not being shown for resources configured in multiple pipelines, due to the globalization of resource caches introduced by v3.0.0.

feature

fly now supports tab completion for -t, thanks to a PR by @jmcarp!

feature

fly set-pipeline now supports tab completion for -c, thanks to a PR by @jmcarp!

feature

Added a timeout when connecting to Postgres.

fix

Fixed running concourse web on Windows. This was caused by building it with an out-of-date version of Go. We've now bumped to the latest.

fix

Bumped the BaggageClaim response header timeout to 10 minutes, up from 1. The point of this value was more to be "not infinite", so that ought to be enough to account for slower disks.

fix

Fixed overriding a base resource type with a custom resource type of the same name.

feature

The legend in the pipeline UI will now show the meaning for dashed vs. solid lines, thanks to a PR by @Typositoire!

feature

The concourse/lite Vagrant box will now respect any locally configured proxy, thanks to a PR by @akumria!

fix

The ATC will uses <a> instead of <span> for a few clickable elements, so that things like Vimperator or Vimium can target them. This is thanks to a PR by @dolph!

feature

The Semver resource will now ignore leading/trailing whitespace in the contents of the file in the bucket, thanks to a PR by @jghiloni! This would usually happen when manually adding the file, as a trailing linebreak is a common default for most editors (or even echo).

fix

The Docker Image resource should now correctly handle fully numeric tags rather than forcing the user to quote them, thanks to a PR by @benmoss!

outdated

v3.0.1

May 16, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.0.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.0.1
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

Fixed a regression in the handling of put steps that have no inputs. Prior to v3.0.0 we would ensure that the directory given to /opt/resource/out would exist even if there are no inputs, so that resources don't have to do a mkdir -p.

We now once again ensure the directory exists, and have coverage to ensure this doesn't happen again.

fix

Fixed a regression in task steps causing their params to not be set in the environment when hijacking. This got lost in a cleanup and is now handled in a much simpler way, and also tested so it doesn't regress again.

fix

The fly set-pipeline command will now exit 0 if the user bails out.

outdated

v3.0.0

May 15, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:3.0.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v3.0.0
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

This release requires an update to your workers. You may want to upgrade them first, actually. If you don't the builds will go orange. But maybe you don't care. Read on for more info.

feature

Life finds a way.

Many many moons ago, in the year 2016, we embarked on a noble goal of refactoring how we do, like, everything. The issue started as More explicit worker, container, volume lifecycles, and came to be known simply as "life". There were many puns. They were great at first. But some people took them too far. @joshzarrabi.

Anyway, we're done now.

With this upgrade, you should notice an overall reduction in container and volume counts across your workers. You should also see a substantial decrease in database queries to Postgres and network calls to Garden and BaggageClaim, as all the container and volume heartbeating is now gone.

If you're interested in the refactor, read on.

The general idea is to switch away from creating containers and volumes willy-nilly and nagging the worker every 30 seconds to keep them around. Instead, we create containers and volumes that are associated to a richer schema such that we don't have to keep heartbeating and know exactly when it should go away. So to keep something like a cache around indefinitely, we just don't destroy it, rather than pinging it all the time.

Building on the richer schema, we are also more able to determine when we can re-use a container. For example, if you have the same exact resource configured across 10 pipelines, that will result in only one check container, rather than 10. This is because there's an abstract notion of a nameless "resource config". We still create one container per team so that fly intercept can't break an entire resource's checking in a multi-tenant environment.

We'll also explicitly remove containers and volumes, rather than relying on Garden and BaggageClaim to kill them once we stop caring about them. This will surface failures to delete in a way that's much easier to notice. This also means that if the ATC goes away for 5 minutes, all its containers and volumes stick around, rather than being mercilessly killed.

A design document for all this will be forthcoming.

feature
breaking

Workers are now versioned. This will allow the ATC to ignore workers that are too old if it requires a new feature or protocol change. The fly workers command will now show the version of each worker and warn you if any are out of date.

Any existing workers you have will be ignored until they are upgraded, so if you upgrade your ATC first, builds that are in flight will fail to resume. If you upgrade the workers first, though, the builds will probably succeed.

feature
breaking

In a task config, you used to be able to configure a URI specifying the rootfs for the underlying container via a config known as "the other image, like, the one you should never use".

We renamed it to something more descriptive and harder to confuse with "the good image, like, the one you should use": rootfs_uri.

feature

When fly intercepting a task that uses image_resource, you will no longer be prompted to intercept the check and get containers for its image. You probably never actually wanted to do that so it was more annoying than helpful. This is something that we got for free as part of the life refactor.

feature

Support for web hooks has landed, thanks to a PR from @mainephd with help from @billimek and others! This was a long-awaited feature that should make many GitHub Enterprise admins very happy.

This feature is implemented in a way that requires no changes from any resources - it "just works". There's no special integration with GitHub, BitBucket, or any hosted offering; you just configure webhook_token in your resource and pass the hook URL to your service.

feature

Building on our fancy new schema, we now make sure to keep the image used by a one-off build around for 24 hours. Previously it would expire, like, whenever. I don't know man. It works now.

feature

We've refactored how auth providers are configured such that all code for a given provider can be defined within a single package. Providers also have an interface to fill out that should cover everything a provider needs to do. This should make it easier (and safer) to submit PRs for auth providers.

feature

The TSA can now be configured with multiple ATC API endpoints to register with, for HA.

fix

Fixed a UI quirk with the sidebar in Firefox, thanks to a PR by @archSeer!

feature

When a resource is failing to check for a meta-level thing like not being able to create its container, the error will now be surfaced in the UI, thanks to a PR by @davidje13!

feature

The Docker Image resource now uses Docker v17.05.0-ce, which notably includes support for multi-stage builds!

feature

BaggageClaim will now do more efficient copying on Windows, thanks to a PR by @jdeppe-pivotal!

fix

The S3 resource can now upload files larger than 5MB to GCS, thanks to a PR by @ljfranklin! This was a complicated intersection of API incompatibilies between IaaSes and assumptions made by SDKs. If you want to relive my confusion, see this comment on the PR.

fix

Concourse looks a bit less jank in Firefox, thanks to a PR by @archSeer which fixes the sidebar padding!

feature

The Hg resource supports a more general revset_filter configuration, thanks to a PR by @cdevienne!

feature

Fly learned the fly teams command, thanks to a PR by @joonas!

feature

The ATC will now emit a worker volumes metric.

feature

The http response time metric now includes the request method, thanks to a PR by @aditya87!

outdated

v2.7.7

May 12, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.7

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.7
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

We forgot to add the flags for configuring InfluxDB. Ignore these release notes and read v2.7.5 and pretend that we didn't mess up.

Thanks.

Again.

Really though: we had done this work on the branch for the upcoming v2.8.0 release, and cherry-picked the work over for v2.7.5 and v2.7.6, but in our haste missed pulling over these changes to the different distribution formats. We've got test suites for this stuff, obviously, but there's not much coverage for metrics and other configurations that are relatively low-risk but costly to test.

outdated

v2.7.6

May 11, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.6

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.6
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

We forgot to add the BOSH properties for configuring InfluxDB. Ignore these release notes and read v2.7.5 and pretend that we didn't mess up.

Thanks.

outdated

v2.7.5

May 11, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.5

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.5
garden-runc release: v1.6.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

The ATC can now be configured to emit metrics directly to InfluxDB. We've also made it easy to extend the ATC with support for more metrics emitters. For examples to reference when implementing your own, see the InfluxDB, Lager, and Riemann emitters.

Note that if you switch from an "ATC -> Riemann -> InfluxDB" stack to "ATC -> InfluxDB" (direct), you'll have to blow away your metrics database. :( This is because when Riemann emitted our metrics it emitted the values as float, and now they're int. The first point emitted to InfluxDB determines the schema, so the new writes do not succeed.

Also note that if you were previously relying on the metrics being emitted to the logs, you'll now need to pass the flag --emit-to-logs. This is now modeled as its own metric emitter and has been made opt-in, as otherwise it could just be producing a lot of useless logs if you've already configured a metrics sink or just don't care.

feature

The ATC will now retry connecting to Postgres when the server-side connection limit is reached. It uses fancy Exponential Backoff Technology™.

fix

Upgrading from v2.7.0 to any of the previous releases would leave the user in a broken state until they logged out and back in. It would show that you're logged in, but any actions taken (e.g. pausing a job or triggering a build) would silently fail. They will now send the user to the log-in page instead.

fix

Previously, with multiple tabs open, logging in to one tab would leave the other tabs in a weird half-logged-in state (due to our CSRF security fix). It should now...not do that.

To the user this may have appeared as being prematurely logged out, because no one really keeps track of which tab is which. We suspect this is the actual root cause.

fix

We've bumped Garden-runC to 1.6.0 which fixes a regression that caused /etc/hosts and /etc/resolv.conf to be unmodifiable.

outdated

v2.7.4

April 26, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.4

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.4
garden-runc release: v1.5.0

See the Concourse BOSH Deployment repo for documentation.

fix

We've bumped Garden-runC to 1.5.0 which fixes a bug which affected the generation of /etc/resolv.conf in some cases.

outdated

v2.7.3

April 12, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.3

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.3
garden-runc release: v1.4.0

See the Concourse BOSH Deployment repo for documentation.

fix

Fix support for postgresql.address in BOSH manifests. Turns out if you type "host and port" long enough you start typing "hort" instead of "host". Don't laugh.

feature

The Git resource now supports pushing with merge: true, which is analogous to rebase: true but works by merging the remote branch into the current HEAD before pushing. This can be useful to preserve the history of your local commits, i.e. if there's a version tag pointing to HEAD and you don't want commits that aren't technically in the tag to be behind it.

feature

The Git resource now supports adding and pushing notes, thanks to a PR by @ahume!

fix

The Git resource, when configured with rebase: true, would previously discard merge commits, losing historical accuracy of the branch. It now preserves them, via --rebase=preserve.

feature

The Docker Image resource now uses a better caching strategy for cache: true, thanks to a PR from @databus23!

fix

The Docker Image resource no longer makes strict assumptions about the format of the repository, supporting repository names like my-repository.biz/a/b/c, thanks to a PR by @ashb!

feature

The Github Release resource resource now supports (and encourages) configuring it with owner, rather than user, which is closer to the GitHub API terminology (and generally makes more sense, since e.g. concourse is an organization, not a user). Thanks @krishicks for the PR!

fix

The Hg resource now explicitly checks against the configured branch. This was a longstanding bug fixed with a lot of patience from @Fydon and assistance from @andreasf - thanks to both!

fix

The Tracker resource previously got 400 errors back when checking for story activity, due to an API change on Tracker's part. This may have been fixed by them by now, but there was also a fix in the resource to not send a trailing ? when the query params are empty.

outdated

v2.7.2

April 11, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.2

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.2
garden-runc release: v1.4.0

See the Concourse BOSH Deployment repo for documentation.

fix

A feature originally intended for 2.8.0 snuck in to 2.7.1 and caused breakage around SSL communication. We've disabled it by default. Sorry about that!

We'll do more extensive testing for this feature by the time it makes it to 2.8.0.

feature

The connection to Postgres can now be configured with SSL. Along the way we've also broken the single opaque --postgres-datasource flag on the web binary into multiple more descriptive flags, which should make it easier to discover what you can or should configure. To see the flags, consult web --help.

Note that the binaries still default --postgres-sslmode to disable for backwards-compatibility. Unfortunately the configuration value of prefer is not available in our Postgres DB driver of choice, so it was either require SSL by default in all configurations (which would be unreasonable for small local deployments) or just leave it off by default.

The BOSH release has always been configured via discrete properties, rather than a single DataSource, and now has a postgresql.ca_cert property among others. Consult bosh.io for more information.

fix

Fixed a couple quirks related to our security fixes that affected folks with colons or any other funny characters in their pipeline names. Moral of the story: never use a regexp if you can help it. Also, y'all have weird pipeline names.

(...Ok, we do too, but I was hoping we were the odd ones.)

outdated

v2.7.1

April 6, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.1
garden-runc release: v1.4.0

See the Concourse BOSH Deployment repo for documentation.

fix
security

If you have a team configured with Generic oAuth, you'll want to upgrade to this release ASAP. Previously a check was missed for this provider in particular that allows users to obtain a token.

We have fixed this hard by making it impossible to forget to update that code path in future PRs. Providers now have a full interface to fill out, rather than having code that implements them strewn throughout the codebase.

fix
security

This fix closes a CSRF security hole in the ATC API. Previously someone could fool you into clicking a link that executes destructive AJAX requests on your behalf. This was possible because the ATC API permits cookie-based auth so that JavaScript EventSource requests (used for streaming build logs) could be authorized.

fix
security

A few headers for security hygiene are now configured on the ATC: X-XSS, X-Frame-Options (configurable; default off so your CI displays will still work), and a few parameters on the cookies that the ATC uses.

fix
security

Previously the fly_local_port query param used in the fly login flow for oAuth could be munged in such a way to send the token to an arbitrary website. This meant that someone could fool you into clicking a link that sent a valid token to themselves. This is now fixed by validating that the query param is numeric.

fix
security

Previously the redirect query param used in the redirect flow for oAuth could be set to any URL, sending you there after you log in to your oauth provider. Now Concourse verifies that the URL is relative to your Concourse domain, and returns a 400 otherwise.

fix

Fixed a memory leak in the TSA. The leak occurred every time a connection was made to Garden or Baggageclaim on a forwarded worker. This took us a while to notice and fix because we're already running the code targeting v2.8.0, which includes a large refactor which results in fewer network calls. Thanks to everyone who helped us dig into this!

fix

The Darwin binary no longer checks that you're running it as the root user. This was initially added because in principle you can run tasks as particular users, but this feature is not well-supported yet, so it's easier to just run the binary as a user that can e.g. talk to Xcode.

We'll add this check back once we properly support running tasks as custom users.

feature

The TSA can now be configured to register against multiple ATCs, rather than just one URL. It will pick a random ATC every time it heartbeats.

outdated

v2.7.0

February 10, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.7.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.7.0
garden-runc release: v1.1.1

See the Concourse BOSH Deployment repo for documentation.

fix

A long-standing bug in Golang's golang.org/x/crypto/ssh package has been fixed. This bug led to workers becoming stuck and/or unregistered after 1GB of data was transferred over their SSH connection. This resulted in builds being stuck in a pending/started state, and resource checking no longer occurring.

This bug affected many people with workers forwarding their registration through the TSA, as is the default for the binary distribution, and is a common configuration for external workers.

For more info, read on.

Context: Workers register via a single long-lived SSH connection. As a baseline, heartbeating and logging goes over this connection, but if the worker is forwarded through the TSA, all API calls and data transfer will also be sent over this encrypted connection (rather than directly to the worker).

The bug: Per the SSH RFC, after some amount of data transfer (1GB by default), a new key is negotiated, so that the connection's encryption has sufficient entropy. The Golang library had a logic error that led to a deadlock during this key negotiation. This led to the connection being "alive" but with the SSH server no longer able to transfer data to and from the worker. This meant API calls would hang, and the worker would eventually unregister as it would fail to heartbeat. The client-side of the worker registration would be stuck waiting for a keepalive response, and so it would never break the connection and recover.

GitHub issues #18439, and later #18711 and #18850 track the journey through debugging and the path to the fix. Thanks to @hanwen for fixing it, and @databus23 for helping keep track of all this!

feature

The fly targets command will now show the team saved for each target, thanks to a PR from @joonas!

feature

The fly login command will now remember the team you were targeted with, making it easier to log back in to the same team and have per-team targets.

fix

Previously if you configured an --external-url using a hostname (e.g. http://some.dns.name:8080) the ATC would have bogus links for the login flow. This has been fixed.

fix

Previously if you were logged in, and then your cookie became invalid but didn't expire (e.g. your session signing key got rotated, possibly via a stemcell update), you wouldn't be able to log in again via basic auth until the cookie was deleted. This has now been fixed.

feature

The ATC's --development-mode flag has been removed in favor of having an explicit flag for --log-level and a flag for --no-really-i-dont-want-any-auth. The BOSH properties have also been updated accordingly.

fix

Previously if the ATC startup was interrupted at an inopportune moment during first-time setup, the internal table for tracking migrations progress could be left in a partially-created unrecoverable state. This has been fixed.

fix

We've fixed an issue in the worker lifecycle wherein a worker that was landing blew up and then tried to come back under a different name, but with the same IP. This could happen if a worker was initially being landed normally, but then a cosmic ray blasted into your infrastructure's datacenter and caused BOSH to recreate the VM instead.

Now, instead of the new worker being unable to register, it'll...be able to register. The old, cosmic-ray-obliterated worker will still be around (under the original name, in landed state), and you'll just have to run fly prune-worker to clean it up.

outdated

v2.6.0

January 5, 2017
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.6.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.6.0
garden-runc release: v1.0.4

See the Concourse BOSH Deployment repo for documentation.

feature

Workers will now, by default, wait for builds to finish before exiting. This will make it safer to perform a rolling update of a Concourse cluster.

If you're running a BOSH deployment, this feature will just start happening automatically. If you're running the binary distribution or Docker, you'll need to invoke land-worker (for a temporary in-place update, i.e. preserving containers and volumes) or retire-worker (for a permanent exit) to initiate draining, and then wait for the worker process to exit.

More docs on this are forthcoming; this release was expedited by the Time resource bug, so docs are sparse at the moment.

feature

Workers that have not heartbeated in a while will now enter a stalled state rather than just disappearing. This should improve resilience to network blips and makes the worker lifecycle much more explicit, allowing us to distinguish between accidentally-unavailable workers and intentionally-removed workers. This way we can continue to retry and wait for the worker to return.

New workloads will not be placed on stalled workers. Stalled workers that will not be coming back can be cleaned up with the new command, fly prune-worker.

fix

The Time resource did not know to compare years. Yep. Pretty silly. So any interval triggers stopped triggering. Time is hard.

feature

Fly learned the fly logout command, which can be used to forget a target and its token. This is thanks to a PR by @mkreibe!

feature

Fly learned the fly validate-pipeline command, which can be used to...validate a pipeline. It does this without needing an external server, either, making it handy for quick local verification or automated testing.

This was a PR submitted by @jmcarp - thanks!

feature

Previously if you configured a job with multiple get steps with the same name, the job would never be able to schedule. @cnelson did a PR to add a validation for this borked configuration - thanks!

feature

fly should now support colors on Windows, thanks to a PR from @alex-slynko!

feature

The BOSH release was changed to default Baggageclaim to consuming (disk size - 5GB), which was all fine and good until your disk was <= 5GB, which caused it to fail. It will now use the full 5GB. Though...you should probably just get more GBs.

feature

Resource checking can now be directed at a tagged worker by specifying tags on the resource.

outdated

v2.5.1

December 13, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.5.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.5.1
garden-runc release: v1.0.4

See the Concourse BOSH Deployment repo for documentation.

fix

Soooo you may have noticed Chrome being really slow lately, especially the autocomplete in the URL bar. We had a bug that led to an infinite redirect loop, causing a bunch of very large URLs to enter the browser history. We've fixed this now. You may want to clear your history to speed Chrome up again. Sorry.

fix

The concourse/lite box will now add Google DNS to the tail of the DNS chain, rather than the head, allowing local DNS resolution settings to be tried first. This is thanks to a PR from @iMartyn!

feature

The Docker Image resource will now propagate the correct --mtu value to the daemon, fixing image fetching flakiness on IaaSes like GCP, which have a default MTU lower than 1500.

fix

When using --ca-cert with fly login, the cert will be appended to the system cert pool, rather than an empty pool. This way the cert will be verified in the case where it's an intermediate cert signed by a root CA in the system pool.

fix

The Git resource can now be configured to force-push, thanks to a PR by @dfedde-pivotal! Use with care.

feature

The Docker Image resource now supports ECR urls in the FROM section, thanks to a PR from @donaldguy!

feature

The build events API endpoint will now return the X-Accel-Buffering header, which hints to reverse proxies to not buffer the response, thanks to a PR from @jasonkeene.

fix

Fixed the janky autorefresh on the job page.

outdated

v2.5.0

November 16, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.5.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.5.0
garden-runc release: v1.0.3

See the Concourse BOSH Deployment repo for documentation.

feature

Teams can now be destroyed via fly destroy-team.

fix

Fixed a hairy deadlock that could lead to jobs getting stuck "waiting for a suitable set of input versions". We fixed it, like, really hard. Like the lock isn't even THERE anymore, man. (And it's not needed anymore, either. That's important too.)

feature

The Cf resource now has the latest CLI version again. Unbeknownst to us, the CLI team switched buckets, so we stopped getting new bits.

fix

We've fixed the CLI download links on the "no pipelines" page.

fix

The fallback flow in login for accepting the token manually is now fixed, thanks to a PR from @sharms!

feature

The Bosh Io Stemcell resource will now aggressively retry downloads, thanks to @zachgersh and @ljfranklin!

feature

The S3 resource now supports setting a Content-Type for the file being uploaded, thanks to a PR from @pdelagrave!

outdated

v2.4.0

October 28, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.4.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.4.0
garden-runc release: v1.0.0

See the Concourse BOSH Deployment repo for documentation.

feature

Worker keys can now be authorized for only a particular team. This prevents workers from being unintentionally (or maliciously) registered as a global worker, in the case where an operator is granting an external worker access to the cluster.

Consult web --help for CLI docs or bosh.io for BOSH docs.

feature

We've lowered the default memory/CPU usage of the concourse/lite Vagrant box to 2GB and 2 cores, down from 6GB and four cores, thanks to a PR from @jwiebalk!

feature

Baggageclaim will now be more durable to corrupt volumes. Previously a borked metadata file would effectively wedge the Baggageclaim API, making the worker unrecoverable. You would see an error like "failed to list volumes" in your builds. Baggageclaim will now pretend these volumes don't exist in the API, and reap them from the disk.

feature

on_failure, on_success, and ensure can now be attached to a job, thanks to a PR from @jmcarp!

feature

fly login will now automatically transfer the token to the CLI for the oAuth flow, rather than requiring you to copy-paste it.

fix

Fixed the behavior of the "home" button. It will now take you to your current pipeline, rather than always taking you to the first one.

fix

After logging in, the UI will now reflect that you're actually logged in. This used to require a refresh. Single page apps giveth and they taketh away.

fix

When viewing a build or a job, the groups the job are in will now be highlighted, rather than always the first group.

fix

Fixed a janky synchronization issues when updating the top bar while switching between pipelines; it used to sometimes show the previous pipeline and never update.

fix

The favicon will now reset back to the default "grey" flavor when switching from a build to any other page.

feature

Logging in will now redirect you back to where you were if it was initiated by some attempted action.

fix

The Bosh Io Stemcell resource now correctly returns versions in chronological order.

outdated

v2.3.1

October 13, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.3.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.3.1
garden-runc release: v0.9.2

See the Concourse BOSH Deployment repo for documentation.

fix

Fixed middle-clicking and other modifier keys when clicking on jobs/resources in the pipeline view.

outdated

v2.3.0

October 13, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.3.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.3.0
garden-runc release: v0.9.2

See the Concourse BOSH Deployment repo for documentation.

feature

The whole UI now runs as a single Elm app! Pages should load much quicker, and the pipeline sidebar now remains open as you navigate around.

There's still some UX work to be done to make things a big smoother, e.g. better handling for 404 cases and more consistent loading indicators, but this is the first big step on that path.

feature

The team name is now provided as $BUILD_TEAM_NAME along with the rest of the metadata available to resources, thanks to a PR from @SHyx0rmZ.

fix

Fixed the log out menu being unclickable on the build page.

fix

The sidebar no longer scrolls offscreen.

feature

The Github Release resource now supports publishing pre-releases, thanks to a PR from @ahelal!

feature

The Git resource can now have LFS disabled via a disable_git_ls param, thanks to a PR from @SHyx0rmZ!

feature

Unused resources in the pipeline config are now a validation error, thanks to a PR from @mmb!

feature

The BOSH release can now be configured with arbitrary Riemann tags, thanks to a PR from @combor!

feature

When configured with a CloudFront endpoint, the S3 resource will now download via CloudFront, which should be much faster. This is thanks to a PR from @cunnie and @ljfranklin!

feature

The S3 resource now supports v2 signature signing, thanks to a PR from @JamesClonk!

feature

The Bosh Deployment resource can now be configured to not redact properties from the deploy diff, thanks to a PR from @jszroberto!

outdated

v2.2.1

September 19, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.2.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.2.1
garden-runc release: v0.8.0

See the Concourse BOSH Deployment repo for documentation.

fix

Finished up the build rendering performance fix on Chrome, which only affected Chrome because Safari and Firefox didn't render Flexbox properly, which is also why their autoscrolling didn't work.

Web. Development.

fix

Fixed autoscrolling in Safari and Firefox.

fix

May have talked up the algorithm release note a bit much. Someone immediately found another case where the 100% CPU monster struck.

This is fixed now. Trust me.

feature

The Bosh Io Stemcell resource has been rewritten in Go with tests and such, thanks to a PR by @zachgersh!

It now does parallel downloads, to boot.

feature

The ATC can now be configured with a Riemann service prefix, thanks to a PR by @mastertinner!

fix

Fixed an issue where the exponential backoff when talking to a flaky worker would never give up.

fix

The Windows fly download link didn't work in the binary distribution. Because of the .exe suffix. Oi. Fixed.

fix

BOSH-deployed workers will now be named after a frankenguid taking parts from the BOSH instance ID and their hostname. This is to make it so you can correlate the worker to the BOSH instance, while also guaranteeing that when the worker is recreated it comes back under a new name.

outdated

v2.2.0

September 15, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.2.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.2.0
garden-runc release: v0.8.0

See the Concourse BOSH Deployment repo for documentation.

feature

The Algorithm has become much faster. The Algorithm is what computes the candidate set of inputs for a job, and is the second hardest problem in Concourse (behind the pipeline UI).

In some cases, e.g. when disabling an oft-used resource version, The Algorithm would go buck-wild and use 100% CPU trying to locate the new set of version candidates. This was undesirable.

Luckily, we prepared for this kind of degenerative case, and made it easy to capture the data sets that replicate the issue. We captured the data set, observed the slowness (had a build running for >12 hours before we gave up), thought long and hard, and did a bunch of work to bring that down to ~19 seconds.

Hopefully that's the last of the 100% CPU monster. Overall scheduling performance has also improved across the board.

feature

The pipeline view will now only redraw if the data has changed. We've also fixed a regression in v2.0.0 that led to redrawing multiple times on an interval, likely leading to the tab crashing if left in the background.

feature

Turns out fly set-team made it stupidly easy to configure a team (or reconfigure an existing team) with no auth credentials. I'm not saying something bad happened, but uh, it'll now warn you and force you to type a really long flag, and even shame you a little bit even when you use it.

feature

The fly binaries are now build natively on each platform, rather than cross-compiled. This removes a few surprises like native DNS and OS X Keychain functionality not working.

They're also now available for download alongside the rest of Concourse, rather than having to download from a Concourse installation.

feature

The --auth-duration flag introduced in v2.1.0 is now available as a BOSH property (auth_duration, surprise surprise). Thanks to @JamesClonk for the PR!

feature

fly checklist now generates a Checkfile with the team name present, thanks to a PR by @Amit-PivotalLabs.

Be sure to upgrade Checkman as well for this to work.

feature

The Generic oAuth provider now supports checking presence of a scope, thanks to a PR by @LinuxBozo!

feature

The Docker Image resource now supports build args, thanks to a PR from @o-orand!

feature

fly sync will bail early if the versions already match, thanks to a PR from @geofffranks!

fix

Turns out Chrome is really, really bad at rendering our build page now. We've made some improvements to this but I think more work is ahead of us.

Compared to Firefox and Safari, Chrome seems to redraw the entire dang page on every friggin update. Which means every second when we update that stupid little ticker up top, the whole page and all its output repaints.

For shame, Chrome.

If it's unbearable you can try Firefox or Safari, which seem to render more sensibly.

We've also fixed a bug that led to interpreting the event stream multiple times for builds that have a ton of output, thereby making things even slower and jankier.

fix

The Docker Image resource skip_download parameter now works again. This broke in v2.1.0. Sorry about that.

outdated

v2.1.0

September 8, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.1.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.1.0
garden-runc release: v0.8.0

See the Concourse BOSH Deployment repo for documentation.

feature

We've reduced the number of queries by about 60%, including removing constant write loads which may have led to increased CPU usage on RDS.

feature

The resource page is now much much more responsive. We've rewritten it in Elm, implementing live-updating along the way. It used to take a few (maybe quite a few) seconds to load, and now takes on the order of milliseconds. Pretty rad.

feature

Triggering a build will now update the UI in-place rather than redirecting.

fix

Autoscrolling is back and better than ever before. Keyboard controls for scrolling (e.g. Cmd+Down, Spacebar) should also now work as normal.

fix

We've updated to Go 1.7.1, which should fix a few DNS-related quirks.

fix

Cmd-click and other non-vanilla clicks should now work for build links in the header of the build view.

feature

The ATC can now be configured with a --auth-duration flag, making the duration for which tokens are valid configurable. This is thanks to a PR from @fmy!

feature

The Git resource now supports GPG verification for commits, thanks to a PR from @alext!

feature

The Docker Image resource now emits the output of docker inspect <image> as docker_inspect.json, thanks to a PR from @endzyme!

feature

Concourse now rejects traffic from web crawlers by providing a robots.txt. We may make this a bit more targeted in the future, but the intent is to reduce unwanted traffic as there are many many many links to click in Concourse.

feature

Thanks to @databus23 the Docker Image resource can now cache things once again! This regressed with Docker 1.10 as the semantics for caching and layer reuse changed to require some additional work/metadata.

feature

The Git resource now supports [skip ci] in addition to [ci skip], thanks to @fmy!

outdated

v2.0.2

August 30, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.0.2

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.0.2
garden-runc release: v0.6.0

See the Concourse BOSH Deployment repo for documentation.

Turns out when you wait a month between releases a few things can go wrong once you finally ship. This is, like, probably the last patch on v2.0.0. Maybe. We'll see. We may save our pride and release v2.1.x next irregardless.

Luckily we have this pipeline thing that lets us continuously fix our own mistakes, not just ship them!

fix

Previously a cluster of multiple ATCs could get into a deadlocked state when checking for resources. This would manifest itself as jobs being stuck in a "pending" state. This release, our first ever X.0.2 release, fixes that.

outdated

v2.0.1

August 29, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.0.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.0.1
garden-runc release: v0.6.0

See the Concourse BOSH Deployment repo for documentation.

fix

Previously if you were using Safari the pipeline would not render. Well, technically, it would render, but within a <div> element with 0px height. We have sighed, flailed at the CSS monster, triggered our web-development pipeline, and prevailed.

Also the top bar used to shrivel up and die if the size of the content page became too large. It, uh, doesn't anymore.

fix

We've restored the pre-teams API endpoint for the job status badges, so you all don't have to update your READMEs immediately. Sorry about that. (You should probably still update them, though.)

fix

We've improved the error message returned when the file used by a task step does not exist.

outdated

v2.0.0

August 26, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:2.0.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v2.0.0
garden-runc release: v0.6.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

TEEEEEEEEEEEEEEEEEEEEEEEEAMS!

So, you'll notice that version number made quite a jump. This is why. The long-awaited "teams" feature brings (trusted) multi-tenancy to Concourse.

The following breaking changes have been made:

  • The --publicly-viewable configuration is gone, and is now set on a pipeline-by-pipeline basis, via fly expose-pipeline and fly hide-pipeline. Newly configured pipelines are hidden by default, and all existing pipelines will be hidden upon upgrade, so make sure to expose the ones you intend to be public!

  • Many API routes now require the team to specified in the URL, e.g. /api/v1/teams/foo/pipelines. Our API still not yet an official interface to Concourse; we continue to encourage using fly until we turn it into a properly versioned and documented API.

    The web UI routes now also have the team name in them. Old URLs should continue to work, and will now redirect to the new URL.

Everything else (pipelines and such) should continue to work the same as before, only now they'll belong to the main team.

feature
deprecation

We have deprecated the /builds page, previously known as "the second hamburger menu button". Its button in the header was taking up valuable UI space, so we unceremoniously demoted it.

Many of you feel bad for this page. That is because you crazy. It has no feelings! fly builds is much better.

Before we kill it off completely, we'll make sure fly builds does everything you'd need from the page, which is really just a matter of having a column for the build URL so you can view it in the web UI. Aside from that, fly builds is better in every way: if your next step is fly intercept, it keeps you in the terminal. You can also change the number of results with -c, and filter it to a job with -j, both of which should be much faster for finding what you're looking for.

feature

The favicon will now change color when viewing a build to reflect its status. Pretty neat, right?

Thanks to @zachgersh and @rmasand for the inspiration! - <3 @kimeberz

feature

As an incremental step in our march towards a fully Elm-based single-page app, we've made some parts of the navigation much snappier. Switching between builds of a job will now update the UI in-place, and toggling pipeline groups now immediately re-renders the existing data set, rather than reloading the entire page.

We'll be focusing more in the upcoming weeks on bringing more of the web UI up to par, ultimately resulting in one big snappy single-page app (hopefully with none of the gotchas that made me hate them at first, i.e. inconsistent data that forces a page refresh and distrust of the entire app as a result).

fix

Previously if the database or network became sluggish, ATC's locking mechanism would stop functioning, resulting in multiple ATCs trying to manage the same build, among other things. We've switched to Postgres session locks, which should be much more airtight.

feature

The ATC now supports being configured with a generic oAuth provider. Huge thanks to @poida for doing the PR for this!

feature

The ATC now supports being configured with a UAA/CF auth provider.

feature

The Docker Image resource can now discover older versions. Previously it would only ever emit the current version. This can be used to roll back to a previously known-good image digest.

feature

We've bumped to Go 1.7 everywhere, and made this process continuous. Whenever Go 1.7.1 or 1.8 come out, we'll automatically pick it up. Turns out there's this pretty neat CI system that can do that kind of thing. You may have heard of it. (It's not Jenkins.)

feature

Fly learned the fly pause-resource and fly unpause-resource commands, thanks to pull requests from @gregarcara!

feature

The bar along the top of the page will now turn blue on already-rendered pages if the pipeline the page belongs to is paused.

feature

The fly login command now accepts a --ca-cert flag, which should be used instead of -k. The cert will be persisted for the target (even if its file goes away).

fix

We've refactored the internal scheduler component of the ATC, reducing query usage and generally making it easier to work on in the future.

This refactor also resulting in fixing behavior with version: every.

fix

Fixed volume deletion in BaggageClaim on a few platforms. May have been primarily situations where the root disk was btrfs.

feature

The S3 resource now supports encryption options, thanks to a PR from @jmcarp!

feature

The Github Release resource now creates a body file when fetching a resource, thanks to a pull request from @shinji62.

Now you can continuously read Concourse's release notes with to determine whether to auto-update!

fix

The Docker repository image now bakes in the default CA certs, thanks to a PR from @billimek!

fix

The Git resource's handling of merge commits now makes a lot more sense.

Previously, a merge commit would result in the history of the merged branch showing up in the version set. For tools like git log this makes sense, but from a CI standpoint, you only really care about the effect on the branch that it was merged into: it's all or nothing. This is now fixed, and only the merge commit itself will be yielded as a version.

Shout-out to @chipx86 for helping us reason through this on GitHub!

fix

The Time resource works now.

It was pretty broken before, because time is hard.

There were a couple issues:

  • If your start and stop were configured in some non-UTC timezone, say, -0700, it straight up wouldn't work if the times were late enough in the day. For real.

  • If you configured days and start and stop, the days would be treated as UTC, rather than respecting the timezone in start or stop. What's more, start and stop could be emitted, leaving there no place for a location for the days to even be specified.

    We've added a location field, which should be used instead of embedded offsets, and then days will respect it.

fix

The BOSH release will now leave 10GB of space free for the system, rather than allocating all of it for BaggageClaim. This is mainly to make the failure mode better. Without this overhead, BaggageClaim would fill up the host's disk, then fail to write to that, and then panic and go read-only, making it unrecoverable. Now the BaggageClaim volume will still fill up, but it'll at least be able to expire volumes and such, and the host machine will still function within its 10GB overhead.

This is all thanks to some sleuthing and a pull request from @alext.

feature

The Bosh Deployment resource now has the BOSH cli v1.3262.4, thanks to a PR from @alex-slynko!

outdated

v1.6.0

July 25, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.6.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.6.0
garden-runc release: v0.4.0

See the Concourse BOSH Deployment repo for documentation.

feature

We now provide an official Docker repository at concourse/concourse!

As part of this, the binary distribution has been updated to support environment variables for configuration, in addition to flags. Because the environment is perfectly safe.

Thanks to @gregarcara and @MeteoGroup for maintaining Concourse images until we started on this ourselves!

feature

The bosh.io Release resource will now verify SHA1 checksums, and place them in the fetched directory as sha1. The bosh.io Stemcell resource has also been updated so that they both have the same behavior.

feature

The Docker Image resource now supports ECR! There were a couple issues and pull requests opened for this; thanks to all who kept the ball rolling!

outdated

v1.5.1

July 20, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.5.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.5.1
garden-runc release: v0.4.0

See the Concourse BOSH Deployment repo for documentation.

fix

A bug introduced by v1.5.0 as part of the resource fetching synchronizing led to hanging get steps. It affected resources with large values in source or params. It is now fixed.

feature

A task can now specify the user to run the process as by configuring run.user in run.

outdated

v1.5.0

July 20, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.5.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.5.0
garden-runc release: v0.4.0

See the Concourse BOSH Deployment repo for documentation.

feature

When connectivity to Concourse is lost on the pipeline page, a fancy warning message will be shown.

This started as a PR from @fmy - thanks!

feature

Loading the logs of a build is now much faster (up to 12x improvements have been observed). Rendering performance is unchanged, but we found that for chatty builds the bulk of the time was spent simply downloading the logs.

feature

We will now only fetch a given resource (including image_resource) once per worker. Previously they would all fetch concurrently and each populate the cache, which would storm the worker with network traffic and CPU load. Now one will start fetching and the rest will wait.

feature

We will no longer create no-op containers for cache hits. This should reduce the number of overall containers used by the pipeline.

fix

The build view was only showing the last 100 builds. And none of you noticed! It'll show all of'em now.

fix

BOSH-deployed workers' names will be set to their BOSH instance ID, rather than their hostname. This should make identifying them a bit easier.

fix

The Docker Image resource will now correctly handle private registry URIs without their port included.

fix

We now limit the total number of database connections to 64 per ATC, and have removed a debugging utility that led to deadlocks when a connection limit was reached (and also may have led to those connection limits being reached in the first place).

outdated

v1.4.1

July 8, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.4.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.4.1
garden-runc release: v0.4.0

See the Concourse BOSH Deployment repo for documentation.

fix

A bug introduced by v1.4.0 caused custom resource types that override worker-provided resource types (e.g. git, s3, docker-image) to lead to containers being created repeatedly until your workers couldn't take anymore.

Fixed. Our bad.

fix

The TLS redirecting feature introduced as part of v1.3.0 made fly execute work only 50% of the time when running two ATCs. With three ATCs it would work 33.3%, repeating of course, of the time, and so on.

fly execute now works 100% of the time.

fix

The commit message format in the Pool resource has been once again tweaked so as to not incorrectly trigger GitHub's issue reference syntax, thanks to a PR from @geramirez.

outdated

v1.4.0

July 6, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.4.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.4.0
garden-runc release: v0.4.0

See the Concourse BOSH Deployment repo for documentation.

feature

We've revamped our container retention configuration.

Previously, containers used by failed builds would stick around for 1 hour, and containers for succeeded builds would stick around for 5 minutes. This was pretty dumb. It meant if you had frequently failing builds, containers (and disk usage) would pile up, and if you had a build that failed overnight, you wouldn't be able to investigate anything in the morning.

Instead, as long as the most recent build of a job is failed or errored, we'll keep it around indefinitely. It will be let go as soon as a new build finishes successfully, or fails, in which case that build will be retained instead.

fix

We've fixed a hairy issue that resulted in artifacts sometimes disappearing in the middle of a build. This issue primarily affected users with more than one worker.

fix

The new container retention semantics also fix the "volume mounted to container is missing" bug with hijacking.

fix

We've bumped the version of the Go AWS SDK used by the S3 resource. This should fix some issues related to long-running uploads and downloads.

fix

fly sync now shows a progress bar. You're welcome.

fix

Some of y'all with BIG DATA had volumes too large to fit their reported size in the database. That should work now.

feature

We've bumped to Garden-runC v0.4.0, which should fix the iptables "resource temporarily unavailable" error.

fix

We've gone back to a safer method of killing container process when aborting a build. We had initially switched to signalling the parent process and then killing it if it didn't exit after 10 seconds, however in a lot of cases this would just result in things not exiting when the process tree is sufficiently complex. This also resulted in the Pool resource not giving up in its attempt loop when aborted.

fix

Previously if a worker left the pool at an inopportune moment, Concourse would forget about its volumes, which led to things getting into a wedged state. This is now fixed. You should never have to pause your pipeline to "let it breathe" again.

outdated

v1.3.1

June 16, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.3.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.3.1
garden-runc release: v0.3.0

See the Concourse BOSH Deployment repo for documentation.

fix

Bumping Buildroot brought in git version 2.8.2, which breaks handling of nested submodules. We've moved ahead to master of Buildroot which bumps git to 2.8.3, which should fix the issue.

feature

The Github Release resource resource will now retry on failed uploads, up to 10 times.

fix

The build numbers made in automated commits to the Pool resource are now escaped with backticks so that GitHub doesn't auto-link them to bogus issues. Thanks @geramirez!

outdated

v1.3.0

June 13, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.3.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.3.0
garden-runc release: v0.3.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

We have switched Garden backends to Garden runC. This new runC-based backend has proven in our testing to be far more portable, allowing our binaries to work on just about any stack that's using a recent enough Linux kernel (3.19+).

As part of this upgrade, your existing workers will need to be recreated.

With BOSH, you can do this with bosh deploy --recreate when deploying the new releases.

For the binaries, you'll need to stop the old worker, nuke the --work-dir, and then start the new one.

In addition, we now explicitly manage all aspects of container images. This should dramatically reduce disk usage on your workers, as there's no longer a redundant copy from importing the image into Garden's graph, as long as you're using image_resource. This also means we're now using btrfs for the whole stack, which makes running Docker in Concourse tasks much easier.

As part of this, the binary distribution no longer supports image in the task config. Supporting it has always been a portability nightmare, and we've been discouraging use of image for some time now.

feature

Jobs can now be configured with build_logs_to_retain, which is a number indicating how many builds for which to keep the build output. All build logs except for the most recent N builds will be reaped. You can flip this on for already-existing jobs with thousands of builds and we'll slowly reap them in batches.

feature

A task step in a plan can now be configured with an image field specifying an artifact source to use. This allows for build-and-test flows, where your pipeline produces an image and then propagates the exact image to a task that uses it as its rootfs.

feature

fly volumes now includes much more information about each volume, including its disk usage. This should help track down what's using so much disk, and whether you really just need more space to accomodate your workload.

feature

A Hg resource is now included as part of the core distribution.

feature

When a build is stuck "waiting for a suitable set of input versions", it will now show what input it cannot find versions for, and why.

feature

Previously workers could end up with very poor balancing of containers, in the worst case resulting in one worker handling the bulk of the resource checking load. We now balance checking across workers over time, by only reusing the check containers for up to an hour.

feature

The ATC itself can now be configured to listen with TLS, rather than relying on an upstream component like HAProxy or an ELB for SSL termination.

This also means the ATC can handle HTTP/2 traffic, thanks to Go's magic net/http package. We've seen noticeable speed boosts in the web UI from this alone.

When TLS is configured the ATC will redirect any non-HTTPS GET and HEAD requests to HTTPS.

feature

In addition to HTTP/2, we've done some optimizations that make the pipeline UI much faster and more responsive.

fix

fly intercept's help text now indicates that you can run an arbitrary command.

feature

The Git resource now includes branches and tags in its metadata for each commit.

fix

Previously the Time resource would accidentally report two versions within the boundary of a time range configured with start and stop. This has been fixed.

feature

The Docker Image resource can now be configured with SSL CA certs to trust when communicating with the registry. This allows you to use private registries securely, rather than listing the address as insecure.

fix

The Git resource will now detect the full history of tags when configured with tag_filter, rather than just the latest one.

feature

All core resources now include bash in their image, which should make hijacking more pleasant. We also stripped out extra stuff from some resources, so on the whole the resource images should be a bit smaller.

feature

We've bumped all core images to Buildroot v2016.05, and are now continously integrating with Buildroot.

feature

The Git resource can now be configured to NOT skip commits with [ci skip] in them, thanks to a PR from @zachgersh and @ryanmoran. This is useful when you're pointing at commits of an external repo with an unrelated CI.

feature

The Git resource, S3 resource, and Semver resource now support basic auth when talking to Git repos, thanks to PRs from @MatthiasWinzeler and @JamesClonk.

feature

The Docker Image resource can now be configured with a registry mirror, thanks to a PR from @gregarcara.

outdated

v1.2.0

April 27, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.2.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.2.0
garden-runc release: v0.337.0

See the Concourse BOSH Deployment repo for documentation.

feature

fly learned the fly check-resource command, which allows you to force detection of versions, notably those in the past. This is useful if you've configured a new resource but want to use a version that's not the latest one.

As part of this change we've slightly tweaked how check works (in a backwards-compatible way). Your check script should now include the requested version in the response if it's still valid. This is so that you can run check-resource with the version that you want, rather than the one before it.

feature

get steps can now be pinned to a specific version.

Example:

plan:
- get: my-repo
  version: {ref: cb0ed22c4cfc6b7524bcafc1664b2d27035521f9}

This will lock the my-repo step to the specified version. Note that the version must be valid, must be collected in the resource's version history (which means you may want to use fly check-resource), and must also satisfy any passed constraints listed on the step.

See version for more information.

feature

get steps can now be configured to run with every version of its resource, rather than skipping to the latest.

Example:

plan:
- get: pull-requests
  version: every

This will allow the build to run with every version of the resource, which is probably a bad idea for certain git repos (where folks may push 100 commits at once), but can make a lot of sense for other things (security auditing, handling all pull requests, processing commits across multiple branches, etc.).

See version for more information.

fix

We've fixed the rendering of multi-field versions in the UI to be substantially less confusing.

As part of this we've tweaked how we render steps in the UI. The checkboxes are now more subtle and less button-like, and aggregate steps look cooler.

fix

The ATC now validates that its URL flags are valid URLs. Previously you could configure an --external-url of example.com, which is missing the scheme, so some things would break.

feature

We've bumped to Go 1.6.1. You probably don't care.

fix

The Docker Image resource now requests the correct schema version of manifests from the registry, which should fix cases where it would pull the wrong digest.

This is thanks to a PR from @databus23.

fix

The S3 resource now issues a shouty warning if you're still using from and to.

It's configured to blink but our web UI doesn't (YET) support blinking text. Consider this a warning. You have one release to comply or be met with red, blinking text in your builds.

fix

The Semver resource can now be configured with an identify for the commits made with the git backend, thanks to a PR from @shinji62.

feature

fly trigger-job now has a -w flag for watching the build that was created.

fix

fly now respects $http_proxy and $https_proxy for communication to the Concourse server, thanks to a PR from @ArthurHlt.

feature

The Docker Image resource now has a tag_as_latest param for tagging the image with latest, in addition to any specified tag, thanks to a PR from @shinji62.

outdated

v1.1.0

April 14, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.1.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.1.0
garden-runc release: v0.337.0

See the Concourse BOSH Deployment repo for documentation.

feature

Workers can now configure proxies to use for containers that are spun up on them.

If you're using the binaries, all you have to do is set the standard $http_proxy, $https_proxy, and $no_proxy environment variables. There are also equivalent flags you can pass to concourse worker, which were added for discoverability's sake.

If you're using BOSH, just set the http_proxy_url, https_proxy_url, and no_proxy properties on the groundcrew job.

feature

A task's run can now specify the working directory by setting run.dir.

feature

fly learned the fly targets command, which, surprise surprise, lists the currently saved targets.

fix

The blackbox job in the BOSH release will now once again emit logs, by autodiscovering them from /var/vcap/sys/log/*/*.log.

fix

Fixed rendering of leading whitespace on lines of output in build logs.

fix

Fixed the scrolling behavior of the pipelines sidebar list to not cut off the last couple of entries.

fix

The Docker Image resource is now durable to resource images that do not contain a /etc/password file.

fix

Previously renaming a pipeline made bad things happen to the automatic resource checking and scheduling for said pipeline. Instead of doing this it now renames the pipeline and the pipeline continues to work.

fix

Previously a put step occurring at the start of the plan would not have its source directory created (as there were no artifacts), which would cause some resources to break. We now ensure this directory exists.

fix

You can now scroll up more easily when viewing a finished build. You are all free now!

feature

You can now run fly help and it'll show its help text instead of "unknown command."

fix

Previously if you had an entry in Resource Types and Resources with the same name the ATC catch on fire. It now doesn't.

fix

Users who are present in more than 30 GitHub organizations and/or teams can now authorize with Concourse. You should be rewarded for your popularity.

fix

Piping input into fly intercept will now send an EOF when the input is exhausted (e.g. echo foo | fly intercept ... cat).

outdated

v1.0.0

March 29, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:1.0.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v1.0.0
garden-runc release: v0.335.0

See the Concourse BOSH Deployment repo for documentation.

We made it!

This release, although relatively small on its own, is built on years of feedback and iteration. So these notes will be a bit more broad and cover all the things you may have missed since you last checked in on our little CI system.

First off, a huge thanks to Pivotal for sponsoring our project and letting us work on it full-time. Over the past year we've had 17 team members rotating through, including 2 designers. Pretty sweet.

With 1.0.0 comes a more rigid release policy on our end. You may see deprecations here and there, so keep an eye out for those via the tags next to each release note, but nothing should change backwards-incompatibly until 2.0.0. We'll still be releasing at the same cadence as before, so we'll probably end up at v1.23.0 pretty soon.

Here's a text-form 80's montage of all the things you may have missed since v0.17.0, our first release:

feature

Steps replaced the old style job config.

feature

A standalone binary distribution of Concourse has been introduced. (Download links to the right.)

feature

Caching and more efficient artifact propagation: resources fetched by get steps are cached on the workers and efficiently propagated throughout steps in the build plan.

feature

A single Concourse can be configured with multiple pipelines dynamically.

feature
feature

Custom resource types can now be added via Resource Types in the pipeline, rather than reconfiguring your workers.

fix

Lots of performance improvements and optimizations, and resilience to flaky networks.

feature

The fly CLI has been entirely rewritten and is much more consistent in UX.

feature

Concourse knows its own version number and will warn you if your CLI is out of date.

feature

Tasks have explicit inputs and outputs, making artifact consumption and production a lot easier to follow.

feature

A new color scheme that's more colorblind-friendly.

feature

A whole bunch of improvements to core resources.

...and now for the actual 1.0 release notes, if you're upgrading from v0.76.0:

feature
breaking

The Concourse BOSH release is now built for BOSH 2.0. You will need a recent director to upgrade.

fix

Resources backed by a resource type defined in Resource Types will now periodically check for new versions of the resource type and use the latest one for checking. Previously the same container would be reused forever even if a new version of the resource type was released.

feature

We've added aria-label attributes to all buttons in the UI, which should improve accessibiltiy for folks using screen readers. Still a ways to go overall, but this is a start.

fix

Lots of dots in sequence in build output will now word-wrap once again.

feature

The BOSH release can now be configured to use GitHub enterprise endpoints for GitHub auth.

fix

Connections from ATC to Baggage Claim will now retry on connection errors.

fix

Fixed an issue where volumes would "expire" even though a build was still using them. Did a bunch of refactoring and now it should all be pretty airtight.

fix

We've fixed a goroutine leak on the ATC which would occur every time image_resource was used.

outdated

v0.76.0

March 23, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.76.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.76.0
garden-runc release: v0.335.0

See the Concourse BOSH Deployment repo for documentation.

fix

v0.75.0 introduced a client-side limit of 64 connections to the database, which no one would ever hit so we didn't bother putting it in the release notes. Then a bunch of people with large deployments hit it and their Concourse went cold. Sorry.

We're removing the limit and are going to do some investigation into the ATC's connection pool characteristics before considering adding it back.

feature

The pipelines sidebar is now scrollable. Some of y'all had a lot of them and got tired of buying larger monitors.

feature

Jobs can now have their manual triggering disabled, via disable_manual_trigger.

feature

The BOSH deployment resource now supports BOSH 2.0 manifests. Previously it would explode instead.

feature

The ATC can now be configured to authenticate against a GitHub Enterprise deployment, thanks to @aequitas!

fix

Cleaned up some internals to fix the root cause some noisy but harmless log lines (failed-to-lookup-ttl).

feature

The Semver resource now supports OpenStack Swift as a storage backend, thanks to @ChrisPRobinson!

feature

The Time resource can now be configured to only yield new timestamps on certain days of the week, thanks to @joek!

feature

fly learned the fly rename-pipeline command, thanks to @zachgersh!

feature

The Docker Image resource should now be more durable to flaky Docker registries, by retrying with exponential backoff on network errors or 5xx responses.

feature

The BOSH deployment resource now downloads the deployment manifest when used as a get step.

fix

Previously the Pool resource would require you to specify retry_delay in nanoseconds, which was a bit silly. It now accepts Go duration format, e.g. 30s.

fix

The Tracker resource now correctly handles rejected stories by only delivering them if a new commit has been made after they were rejected.

outdated

v0.75.0

March 9, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.75.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.75.0
garden-runc release: v0.334.0

See the Concourse BOSH Deployment repo for documentation.

deprecation
feature

The Semver resource now creates a file called version containing the version number, making it consistent with other resources that provide a version.

We still create number for backwards compatibility, but you should switch.

deprecation
feature

Specifying both file and config on a task step is now deprecated. You should receive warnings when running fly set-pipeline and when running a task that specifies both.

Instead, you should be specifying params, input_mapping, and output_mapping.

feature

fly and the web UI now know their own version! We probably should have done this years ago. So fly -v now works instead of printing a shrugging emoticon, and the web UI now has the version at the bottom right (it even live updates, for all your CI monitors out there).

In addition, fly will print a warning if the versions are slightly out of sync (patch release), and straight up prevent itself from running if they're significantly out of sync (i.e. minor or major).

fix

Tagged workers are now supported by image_resource.

feature

The duration that containers stick around for after finishing is now configurable via new atc.retention.* BOSH properties (and corresponding flags to the ATC).

feature

fly intercept now sorts its container list, which should aid in frustration with finding the container to intercept.

feature

fly containers now shows the TTL (as we've configured it) and validity (actual expiration, which counts down to 0) for each container. This will be useful to know which ones are sticking around because they failed, and which ones are sticking around because of a build that's running too frequently.

feature

fly learned the fly abort-build command, thanks to a pull request from @zachgersh.

feature

fly learned the fly trigger-job command, thanks to a pull request from @aminjam.

feature

The BOSH deployment resource now supports deploying to a director using UAA client ID/secret for auth.

feature

We've bumped the version of Buildroot that many of our resources are based on, which should bring in updated CA certificates and other miscellaneous things.

fix

Previously resources that had params involving lists of objects would cause Concourse to blow up instead of working. It should now work.

outdated

v0.74.0

February 26, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.74.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.74.0
garden-runc release: v0.334.0

See the Concourse BOSH Deployment repo for documentation.

feature

You can now configure resource types in your pipeline rather than redeploying your workers with additional resource types. This should make it much easier to use the community resources that people have built!

feature

Autoscroll on a build page is back and is now implemented in a way that doesn't kill the browser when you have many build events.

feature

If the Docker image you specify in the image_resource section of your task has a custom user then we will now respect that when running the task. This user will also be used when hijacking in to a build container.

feature

Hijacked connections will no longer cause connection timeouts at interim load balancers if there is no input or output.

feature

The pipeline graph rendering now has large portions of the computation cached. This should provide a significant speedup and decrease in CPU load when viewing a pipeline.

feature

We've made some tweaks to the ATC's build scheduling that should fix "deadlock" scenarios with serial groups.

Previously, if a pipeline of "A -> B -> C" had all 3 jobs in a serial group, and the builds were enqueued in order of C, B, then A (manually), nothing could ever run, as the scheduling was based on the order of the builds being enqueued, and C would never be satisifed. This is now fixed by collecting inputs and then scheduling only once they're available, so that C never gets scheduled, and so A is able to be scheduled, followed by B, and then C.

fix

The icon font that was broken in Safari by v0.73.0 are now unbroken by v0.74.0.

fix

Interrupting a fly execute that was fetching outputs will no longer panic if you cancel it in the middle.

feature

If you try and trigger a build while you are not logged in then we'll now redirect you back to the build page you were on rather than the main pipeline page.

feature

The Pool resource will now ignore in-place modifications when working out if a log aquisition is still valid.

feature

fly will now print the target it will be interacting with at the start of every command.

breaking

fly default value of the -t flag has been removed. If you're using the VirtualBox distribution then you'll need to start logging in and supplying a target. This is to get people in this habit before they progress to a bigger deployment.

feature

fly has a more sensible timeout and a better error message if it cannot reach the targeted Concourse.

feature

We bumped to Go 1.6. You should see absolutely no change.

outdated

v0.73.0

February 18, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.73.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.73.0
garden-runc release: v0.332.0

See the Concourse BOSH Deployment repo for documentation.

feature
breaking

The fly destroy-pipeline command now runs much quicker. Unfortunately to implement this there's a massive database migration. Expect anywhere from a few minutes of downtime up to a few hours when you upgrade to this version, depending on how many builds you have and how chatty they are. Sorry about that.

There is no way of predicting how long this migration may take for your instance. It depends on the chattiness of your builds and the performance of your database. For a rough approximation: running the migration on the Concourse team's server took 20 minutes to migrate 13 million build events.

You can find out how many build events you have by running the following query against your Concourse database:

SELECT relname, n_live_tup
FROM pg_stat_user_tables
WHERE relname = 'build_events';

The root of the issue is the amount of data in the build_events table. If you don't have many build logs then then you can probably just upgrade and not worry about the rest of this. If the upgrade is going to take too long then you'll need to find some way to reduce the rows in that table.

Before you delete any data from the system you should make sure to take a backup of your database and make doubly sure you can restore it while blind-folded and upside-down in case anything goes wrong.

If you don't care about your old build logs then you can simply run TRUNCATE build_events; before upgrading and the migration will be quick and painless.

Many of you probably do care about your build events but maybe only those which were created in the past X months. If this sounds like a good idea then have I got the SQL query for you! Run this against your Concourse database (change the X to the number of months you'd like to keep):

DELETE FROM build_events
WHERE build_id IN (
  SELECT builds.id
  FROM jobs JOIN builds ON jobs.id = builds.job_id
  WHERE builds.end_time < NOW() - INTERVAL 'X month'
  ORDER BY builds.id
);

This query may take a while to execute but your Concourse can be online the entire time that it is running.

If you want to keep all of your build logs and have a fast migration then I'm sorry, I can't help you. :( Maybe upgrade over a weekend?

breaking

The Docker Image resource no longer produces the docker saved image by default. This is to reduce disk usage when using the resource as an image_resource. You must now pass save: true as part of params on the get step to produce the image file.

feature

Inputs to jobs that are not configured to trigger the job when new versions appear will now be rendered with a dashed line. This makes it easier to see which resources automate the pipeline flow, and which jobs are only ever manually triggered.

feature

A pending build will now indicate why it's pending, via a checklist that appears at the top of the build output. (Yay!)

breaking

The theme selector is gone. So are all but one of the themes. This new theme is the product of our research and your feedback. I'm confident that it's perfect in every way. But let us know if you have any major problems with it.

fix

Improved the caching of resources used for image_resource. Previously if the same version was fetched multiple times on the same worker, we'd keep all of them around so long as they were the latest version. We'll now only keep one.

feature

The fly destroy-pipeline command learnt the -n option which when used will not ask you to confirm the deletion of the pipeline. Useful for scripts. Dangerous for users.

fix

The Docker Image resource no longer worked for images configured with ENTRYPOINTs as of v0.72.0, and ended up running whatever the entrypoint was, with our internal binary tool as an argument. It now works again. Our bad.

fix

The GitHub release resource would have issues when fetching artefacts from S3 via GitHub when using an access token. It no longer has these issues.

feature

The Docker Image resource now supports a dockerfile parameter for specifying a path to the Dockerfile to use.

feature

The Git resource now supports producing annotated tags via the annotate parameter.

feature

The Git resource now supports checking for tag patterns like *-production via the tag_filter source configuration.

feature

The Git resource now includes git lfs.

outdated

v0.72.1

February 8, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.72.1

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.72.1
garden-runc release: v0.332.0

See the Concourse BOSH Deployment repo for documentation.

fix

Fixed a bug that would cause containers used by the new image_resource not to be released.

fix

Removed a uniqueness constraint on the database that checked a huge number of columns which caused PostgreSQL to error for certain pipelines.

outdated

v0.72.0

February 6, 2016
Downloads
lewing@isc.tamu.edu Larry Ewing and The GIMP [Attribution or CC0], via Wikimedia Commons

Linux

concourse amd64
fly amd64
By Rob Janoff [Public domain], via Wikimedia Commons

Mac OS X

concourse amd64
fly amd64

Note: Resources require at least one Linux worker.

By Microsoft [Public domain], via Wikimedia Commons

Windows

concourse.exe amd64
fly.exe amd64

Note: Resources require at least one Linux worker.

Docker

concourse/concourse:0.72.0

See the Concourse Docker repo for documentation.

BOSH

concourse release: v0.72.0
garden-runc release: v0.332.0

See the Concourse BOSH Deployment repo for documentation.

feature
deprecation

We've added a new way of specifying the image for a task. You can now use image_resource which will use Concourse resources to pull the image down.

For example:

image: docker:///ubuntu#14.04

would now become:

image_resource:
  type: docker-image
  source:
    repository: ubuntu
    tag: 14.04

If those keys look familiar that's because they're the same ones you would specify if you were defining a resource in your pipeline. This means that because the Docker Image resource supports private repositories you can now use private repositories for your task images. There are more details in the Running Tasks section of the documentation.

Further improvements to this feature are coming soon!

breaking
feature

Resources now have access to the external URL of the Concourse that they are associated with in the environment variable ATC_EXTERNAL_URL.

The property atc.external_url is now required for ATC to start up.

breaking

In order for a task to make something for a later step in the plan it now must use outputs. Previously the task's entire working directory was made available if outputs were not configured, which led to more network traffic than necessary if you didn't intend for the task to actually propagate anything to later steps.

feature

We now validate task configuration at the start of a task. Sneaky things like typos and overlapping inputs and outputs are now hard errors.

feature

The new default theme concourse is the culmination of our work on color schemes. It will soon take its place as the one and only color scheme.

If you've been changing your theme around, make sure to manually switch to the concourse theme to check it out, otherwise it'll stick with whatever scheme you last selected.

feature

Failed jobs are now displayed with an additional border and a symbol above the column that they are contained in. This change along with the improved color scheme should help the 315,000,000 potential and current Concourse users around the globe who are color blind.

feature

The atc.publicly_viewable flag now applies to the API endpoints as well as the web interface. There was no secret information exposed by the API but potentially sensitive things like job names for unannounced projects were there.

fix

We fixed the bug that would show your entire pipeline in the diff when uploading changes to your pipeline.

fix

New line characters in resource metadata output are now preserved again when viewing them in the web build view.

fix

If a job has both pending and running build we will now show the running build animation on the dashboard rather than the less interesting pending animation.

fix

If you view a job or build in a paused pipeline then the top bar on the page will be blue again.

feature

fly now presents a far more helpful error message if you're not logged in.

feature

The pipeline layout algorithm has changed to draw resources at an appropriate height so that more of the lines are straighter.

fix

The list of containers returned when you try and hijack is now correctly filtered to the build number you ask for.

fix

The ATC doesn't panic anymore if the database disappears in the middle of a build. It still gets pretty mad though so try and keep your database disappearing to a minimum.

feature

The Tracker Resource now understands variations on the word "completed" when scanning commit messages.

fix

ATC will now handle if you supply an atc.external_url with a trailing slash rather than not letting GitHub users log in. A significant improvement if I do say so myself.