Kubernetes certificate expiry: find and renew in time

Expired certificates break clusters quietly: kubelets stop talking to the API server, ingress serves browser warnings, webhooks fail. Three groups matter: control plane and kubelet certificates, TLS certificates for your applications, and webhook CAs.

Check expiry with the command line

  • kubeadm clusters: kubeadm certs check-expiration on a control plane node; renew with kubeadm certs renew all.
  • RKE2 / k3s: certificates are valid for 12 months and are renewed when the service restarts within 90 days of expiry; rke2 certificate rotate forces it.
  • A TLS Secret: kubectl get secret <name> -n <ns> -o jsonpath="{.data.tls\.crt}" | base64 -d | openssl x509 -noout -enddate
  • cert-manager: kubectl get certificates -A shows READY; kubectl describe certificate <name> shows the renewal time and the last error.

Kubelet certificates

Kubelets use a client certificate to reach the API server and, optionally, a serving certificate. With client certificate rotation on (the default in current releases) the kubelet renews its client certificate itself. Serving certificates need serverTLSBootstrap and approved CSRs; otherwise they are self-signed and do not rotate.

cert-manager certificates that do not renew

  • ACME HTTP-01 fails: the solver ingress is not reachable or uses the wrong ingress class.
  • ACME DNS-01 fails: the DNS provider credentials or zone are wrong.
  • Issuer not ready: kubectl describe issuer / clusterissuer shows why.

Renew by hand with cmctl renew <certificate> -n <ns> once the cause is fixed.

Set an alert, not a reminder

With Prometheus, alert on the time left: certmanager_certificate_expiration_timestamp_seconds - time() < 14 * 24 * 3600 for cert-manager, and the kubelet's kubelet_certificate_manager_client_ttl_seconds for kubelet client certificates.

The Kubby way

In Monitoring, the Certificates tab shows days left for every kubelet client and serving certificate, cert-manager certificates that are not ready, and API requests made with client certificates about to expire. Without Monitoring, Kubby still flags TLS Secrets that expire within 30 days or have expired in "Needs attention".

Ask Kubby lists the certificates that expire in the next 30 days next to the Monitoring Certificates tab