1.15.4.4 The AWS Secrets Manager credential manager

Configuration

In order to integrate with AWS Secrets Manager for credential management, the web node must be configured with:

• an access key and secret key, or a session token

• the AWS region that your parameters are stored within.

If no access key, secret key, or session token is provided, Concourse will attempt to use environment variables or the instance credentials assigned to the instance.

The web node's configuration specifies the following:

aws-secretsmanager-access-key: string

aws-secretsmanager-secret-key: string

aws-secretsmanager-session-token: string

aws-secretsmanager-region: string

aws-secretsmanager-pipeline-secret-template: string

aws-secretsmanager-team-secret-template: string

For example, to launch the ATC and enable Secrets Manager, you may configure:

concourse web ... \
  --aws-secretsmanager-region us-east-1 \
  --aws-secretsmanager-access-key AKIAIOSFODNN7EXAMPLE \
  --aws-secretsmanager-secret-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# or use env variables
CONCOURSE_AWS_SECRETSMANAGER_REGION="us-east-1" \
CONCOURSE_AWS_SECRETSMANAGER_ACCESS_KEY="AKIAIOSFODNN7EXAMPLE" \
CONCOURSE_AWS_SECRETSMANAGER_SECRET_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
concourse web ...

A more secure method is to configure an IAM role for your EC2 ATC instance so that credentials are fetched automatically from the EC2 metadata service.

Saving credentials in AWS

It seems to be best to use the 'other type of secret' option and the 'plaintext' entry (otherwise your secrets will be interpolated as JSON) for best results. Make sure your secret locations match the lookup templates exactly; include the leading /, for example.

IAM Permissions

The following is an example of an IAM policy that can be used to grant permissions to an IAM user or instance role. Note that the Resource section can contain a wildcard to a secret or be restricted to an individual secret. In order for the health check to work properly (see Scaling), Concourse needs to have access to the __concourse-health-check secret.

{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Sid": "AllowAccessToSecretManagerParameters",
        "Effect": "Allow",
        "Action": [
            "secretsmanager:ListSecrets"
        ],
          "Resource": "*"
        },
        {
            "Sid": "AllowAccessGetSecret",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:DescribeSecret"
            ],
            "Resource": [
                "arn:aws:secretsmanager:*:*:secret:/concourse/*",
                "arn:aws:secretsmanager:*:*:secret:__concourse-health-check-??????"
            ]
        }
    ]
}

If you wish to restrict concourse to only have access to secrets for a specific pipeline, you can replace "arn:aws:secretsmanager:*:*:secret:/concourse/*" in the example above with

"arn:aws:secretsmanager:*:*:secret:/concourse/TEAM_NAME/*",
"arn:aws:secretsmanager:*:*:secret:/concourse/TEAM_NAME/PIPELINE_NAME/*",

where TEAM_NAME and PIPELINE_NAME are replaced with the team and name of the pipeline in question.

For more information on how to use IAM roles to restrict access to Secrets Manager, review the official documentation.

Credential Lookup Rules

When resolving a parameter such as ((foo_param)), Concourse will look in the following paths, in order:

• /concourse/TEAM_NAME/PIPELINE_NAME/foo_param

• /concourse/TEAM_NAME/foo_param

The leading /concourse can be changed by specifying --aws-secretsmanager-pipeline-secret-template or --aws-secretsmanager-team-secret-template variables.

Note that if Concourse does not have permission to access the pipeline-scoped paths, then credential lookups will fail even for credentials which are stored at the team level.

Scaling

If your cluster has a large workload, in particular if there are many resources, Concourse can generate a lot of traffic to AWS and subsequently get rate-limited.

As long as Concourse has permission to get the value of the __concourse-health-check secret, you should be able to measure an error rate by polling the /api/v1/info/creds endpoint when authenticated as a Concourse Admin.

Depending on your workflow for updating secrets and your reliability requirements it may be worth Caching credentials and/or Retrying failed fetches to mitigate rate-limit-related errors.