Terraform ACME provider full chain certificates

After rearchitecting my Concourse setup a bit to shard web workers I’ve noticed that fly stopped connecting:

λ fly login -t fog --concourse-url https://ci.saragosa.tv:443           (1)
logging in to team 'main'

could not reach the Concourse server called fog:

    Get https://ci.saragosa.tv:443/api/v1/info: x509: certificate signed by unknown authority

is the targeted Concourse running? better go catch it lol

Which was surprising to me since WebUI was working without any issues and I was getting green checks on the security tab of Chrome’s “Developer tools”

The important bit here is that I was using Terraform to generate acme certificates and pass them into GCP’s HTTPS LB:

resource "google_compute_ssl_certificate" "ci" {
  name = "my-ci-project-${random_id.generation.hex}"
  private_key = "${acme_certificate.ci.private_key_pem}"
  certificate = "${acme_certificate.ci.certificate_pem}"

  lifecycle {
    create_before_destroy = true
  }

After some digging I’ve found this issue. Turns out that whatever lib fly is using to verify the certificates requires full chain of trust to be present. Surely enough running SSLabs on my domain showed:

Certificates provided      1 (1367 bytes)
Chain issues               Incomplete

Turns out acme_certificate.certificate_pem doesn’t produce a full chain certificate. Fortunately we can simply concatenate it like so:

resource "google_compute_ssl_certificate" "ci" {
  ...
  certificate = "${acme_certificate.ci.certificate_pem}${acme_certificate.ci.issuer_pem}"
 ...
}

After a brief terraform apply SSLabs showed:

Certificates provided   2 (2541 bytes)
Chain issues           None

And fly was once again working correctly. Hope this helps someone if they run into same issues.

 
2
Kudos
 
2
Kudos

Now read this

Retrieving GCE metadata map

GCE metadata can be a useful tool for getting the information about the running instance. One needs to know exactly where to look for a particular key however. For that, the metadata actually supports a ?recursive=true parameter,... Continue →