Tired of Certificate Warnings

My little cluster has a few front-ends now, argocd, longhorn and the arkade servers. It’s getting a little tiresome having to OK certificate warnings like this:

So I started reading up on cert-manager.

I use letsencrypt and have some experience with setting that up. I’ve also got some experience setting up DNS challenges and auto-cert-renew, but I didn’t really want my little pi-cluster on (or near) the internet. I found this great article about setting up self-signed certs on cert-manager and basically followed it to add Certificates to all my IngressRoutes (thanks Remy).

I’m building a little library of setup scripts for my cluster (so far argocd and longhorn). When I roll the nodes, I run these scripts to set things up. The cert-manager.sh comes first (so that I can generate certs for the frontends of argocd and longhorn). That looks like this:

sandy@bunsen:~/k3s$ cat cert-manager.sh 
#!/bin/bash 

. ./functions.sh 

# Root CA is output as hobo-root-ca.crt - can be imported into chrome 
# or sudo cp hobo-root-ca.crt /usr/local/share/ca-certificates 
#    sudo update-ca-certificates

NAMESPACE=cert-manager

helm install \
  cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --version v1.18.2 \
  --namespace $NAMESPACE \
  --create-namespace \
  --set crds.enabled=true

pod_wait $NAMESPACE

kubectl apply -f hobo-root-ca.yaml
sleep 5

echo "Output root CA to hobo-root-ca.crt"
kubectl get secret hobo-root-ca-secret -n $NAMESPACE -o jsonpath='{.data.tls\.crt}' |  \
   base64 --decode | \
   openssl x509 -out hobo-root-ca.crt

kubectl apply -f hobo-intermediate-ca1.yaml
sleep 5
# check the intermediate cert 
openssl verify -CAfile \
   <(kubectl -n $NAMESPACE get secret hobo-root-ca-secret -o jsonpath='{.data.tls\.crt}' | base64 --decode) \
   <(kubectl -n $NAMESPACE get secret hobo-intermediate-ca1-secret -o jsonpath='{.data.tls\.crt}' | base64 --decode)

The script installs cert-manager in it’s own namespace and installs the root CA and intermediate CA ClusterIssuers:

sandy@bunsen:~/k3s$ more hobo-root-ca.yaml 
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: hobo-root-ca-issuer-selfsigned
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: hobo-root-ca
  namespace: cert-manager
spec:
  isCA: true
  commonName: hobo-root-ca
  secretName: hobo-root-ca-secret
  duration: 87600h # 10y
  renewBefore: 78840h # 9y
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: hobo-root-ca-issuer-selfsigned
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: hobo-root-ca-issuer
spec:
  ca:
    secretName: hobo-root-ca-secret

sandy@bunsen:~/k3s$ more hobo-intermediate-ca1.yaml 
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: hobo-intermediate-ca1
  namespace: cert-manager
spec:
  isCA: true
  commonName: hobo-intermediate-ca1
  secretName: hobo-intermediate-ca1-secret
  duration: 43800h # 5y
  renewBefore: 35040h # 4y
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: hobo-root-ca-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: hobo-intermediate-ca1-issuer
spec:
  ca:
    secretName: hobo-intermediate-ca1-secret

ClusterIssuers basically generate key-pairs and dump the key/crt pair into a kubernetes secret. The cert-manager.sh script dumps the .crt out of the root-ca secret into a file called hobo-root-ca.crt which can be installed in chrome (navigate to chrome://settings/ -> Privacy and Security -> Manage Certificates …). You can also import on linux in general with sudo cp hobo-root-ca.crt /usr/local/share/ca-certificates/; sudo update-ca-certificates

With cert-manager installed and configured, I could upgrade the arcade helm chart. Briefly that looks like this:

sandy@bunsen:~/arkade$ git diff 5cbd3f5e742a8a8f272cb4f1547faa51aa1a216d
diff --git a/Makefile b/Makefile
index fa0ea12..1db1dd6 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ $(strip $(firstword $(foreach game,$(GAMES),$(findstring $(game),$(1)))))
 endef
 
 # Variables
-CHART_VER := 0.1.5
+CHART_VER := 0.1.6
 BUILD_IMAGE ?= mamebuilder
 TAG ?= latest
 SHELL := /bin/bash 
diff --git a/helm/game/templates/certificate.yaml b/helm/game/templates/certificate.yaml
new file mode 100644
index 0000000..e5cf300
--- /dev/null
+++ b/helm/game/templates/certificate.yaml
@@ -0,0 +1,14 @@
+{{- if .Values.certificate.enabled -}}
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: {{ include "game.fullname" . }}
+  namespace: games
+spec:
+  secretName: {{ include "game.fullname" . }}-cert-secret        # <===  Name of secret where the generated certificate will be stored.
+  dnsNames:
+    - "{{ include "game.fullname" . }}"
+  issuerRef:
+    name: hobo-intermediate-ca1-issuer
+    kind: ClusterIssuer
+{{- end }}
diff --git a/helm/game/templates/ingressroute.yaml b/helm/game/templates/ingressroute.yaml
index 6838816..3b73b41 100644
--- a/helm/game/templates/ingressroute.yaml
+++ b/helm/game/templates/ingressroute.yaml
@@ -3,6 +3,9 @@ apiVersion: traefik.io/v1alpha1
 kind: IngressRoute
 metadata:
   name: {{ include "game.fullname" . }}
+  annotations:
+    cert-manager.io/cluster-issuer: hobo-intermediate-ca1-issuer
+    cert-manager.io/common-name: {{ include "game.fullname" . }}
 spec:
   entryPoints:
     - websecure
@@ -14,5 +17,5 @@ spec:
         - name: svc-{{ include "game.fullname" . }}
           port: 80
   tls:
-    certResolver: default
+    secretName: {{ include "game.fullname" . }}-cert-secret
 {{- end }}
diff --git a/helm/game/values.yaml b/helm/game/values.yaml
index 4ed05a1..278754c 100644
--- a/helm/game/values.yaml
+++ b/helm/game/values.yaml
@@ -76,6 +76,9 @@ ingress:
 ingressroute:
   enabled: true
 
+certificate:
+  enabled: true
+
 resources: {}
   # We usually recommend not to specify default resources and to leave this as a conscious
   # choice for the user. This also increases chances charts run on environments with little

I added the certificate template in my helm chart, and then added cert-manager annotations to the IngressRoute and changed the tls: definition.

Push the helm chart change and sync the ArgoCD projects and voila!

I also updated the argocd and longhorn setups to add Certificates to their IngressRoute definitions – so now all the frontends in my cluster can be accessed without the security warning.

-Sandy