Use cert-manager to get port 443/https running with signed x509 certificates for Ingress on your Kubernetes Production Hobby Cluster. cert-manager is the successor to kube-lego and the preferred way to “automatically obtain browser-trusted certificates, without any human intervention.” using Let’s Encrypt.

You need to install Helm first if you do not already have it. Otherwise, check out my article Helm on Custom Kubernetes, especially if you are following along with my Production Hobby Cluster guides.

§2026 Update

Good news up top: cert-manager is still the standard way to get Let’s Encrypt certificates on Kubernetes. It is a CNCF project now, living at cert-manager/cert-manager, and it does exactly what this post describes. The catch is that almost every API name and URL below has changed, so the manifests here will not apply to a current cluster. Here is the modern version.

The install is different. stable/cert-manager and helm install --name are Helm 2 relics, and the stable repo is gone. Use the jetstack chart in its own namespace, and let the chart install the CRDs:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --set crds.enabled=true

The API group was renamed. Everything moved from certmanager.k8s.io/v1alpha1 to cert-manager.io/v1. The ACME endpoints changed too: Let’s Encrypt retired ACMEv1, so use https://acme-staging-v02.api.letsencrypt.org/directory and https://acme-v02.api.letsencrypt.org/directory. And the HTTP-01 solver moved off each Certificate and onto the Issuer. A current staging ClusterIssuer looks like this:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
    - http01:
        ingress:
          class: nginx

The biggest simplification: you rarely write a Certificate resource by hand anymore. Annotate the Ingress with the issuer and add a tls: block, and cert-manager’s ingress-shim creates, stores, and renews the certificate for you:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    - www.example.com
    secretName: example-com-production-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: ok
            port:
              number: 5001

Two more notes. The Ingress API itself moved to networking.k8s.io/v1 (the extensions/v1beta1 used below was removed in Kubernetes 1.22), and cert-manager can now solve challenges for Gateway API routes too, not just Ingress. If you are still on the retired ingress-nginx, see my 413 post for where that is headed.

Archived: the original 2018 post follows. The walkthrough below uses the old certmanager.k8s.io/v1alpha1 API and ACMEv1 URLs. It is kept for the archives and no longer reflects current practice. Use the 2026 Update above for the working setup.

§Install cert-manager

helm install --name cert-manager --namespace kube-system stable/cert-manager

After Helm installs cert-manager you end up with a ServiceAccount ClusterRole, ClusterRoleBinding, Deployment and a couple of Pods named cert-manager-cert-manager in the kube-system namespace. Helm additionally installs three CustomResourceDefinitions for cert-manager (custom resources are not namespaced):

• certificates.certmanager.k8s.io
• clusterissuers.certmanager.k8s.io
• issuers.certmanager.k8s.io

It’s good to know what Helm installed for cert-manager and these three CustomResourceDefinitions represent the configurations we are creating in the next steps.

cert-manager uses either an Issuer or ClusterIssuer to represent a certificate authority. Issuer is bound to a namespace so for our Production Hobby Cluster we will use a ClusterIssuer.

We will setup a letsencrypt-staging and a letsencrypt-prod ClusterIssuer.

§Create a ClusterIssuer

First, we need to have a functional Ingress on our Kubernetes cluster. If you have not done so, check out my article Ingress on Custom Kubernetes.

Create a file called 10-cluster-issuer-letsencrypt-staging.yml for the ClusterIssuer, add the following configuration and change the email address to your own. Note this uses the 2018 certmanager.k8s.io/v1alpha1 API and the retired ACMEv1 URL; see the 2026 Update above for the current cert-manager.io/v1 and ACMEv2 form.

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: default
spec:
  acme:
    # The ACME server URL
    server: https://acme-staging.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: [email protected]
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    http01: {}

Create the staging ClusterIssuer with kubectl:

kubectl create -f 10-cluster-issuer-letsencrypt-staging.yml

Expect output similar to the following:

clusterissuer.certmanager.k8s.io "letsencrypt-staging" created

Check the status of the new ClusterIssuer:

kubectl describe ClusterIssuer

Under the status section you should get output similar to the following:

  Conditions:
    Last Transition Time:  2018-05-18T08:05:09Z
    Message:               The ACME account was registered with the ACME server
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready

Now we can create a production ClusterIssuer, for use only after we test with our staging configuration; otherwise, we are likely to hit rate limits while testing.

The only essential difference between the staging and production ClusterIssuer is the server: URL.

• Staging: server: https://acme-staging.api.letsencrypt.org/directory
• Production: server: https://acme-v01.api.letsencrypt.org/directory

Create a production ClusterIssuer with the configuration below, make sure to change the email address to a valid account. The configured email address may receive messages from Let’s Encrypt.

Create a file called 20-cluster-issuer-letsencrypt-production.yml for the ClusterIssuer, add the following configuration and change the email address.

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
  namespace: default
spec:
  acme:
    # The ACME server URL
    server: https://acme-v01.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: [email protected]
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-production
    # Enable the HTTP-01 challenge provider
    http01: {}

Create the production ClusterIssuer with kubectl:

kubectl create -f 20-cluster-issuer-letsencrypt-production.yml

Ensure both ClusterIssuers are present:

kubectl get ClusterIssuer

Output:

NAME                     AGE
letsencrypt-production   3m
letsencrypt-staging      5m

§Obtain a Certificate

Create a file named 30-Cert-DOMAIN.yml, replacing DOMAIN with your domain. Copy the below configuration and change all the example domain references to your own. This configuration generates a test certificate from Let’s Encrypt’s staging environment.

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: example-com
  namespace: default
spec:
  secretName: example-com-staging-tls
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  commonName: example.com
  dnsNames:
  - www.example.com
  acme:
    config:
    - http01:
        ingressClass: nginx
      domains:
      - example.com
    - http01:
        ingress: my-ingress
      domains:
      - www.example.com

The two http01 sections under acme: demonstrate using the http-01 challenge with and without an existing ingress. The http-01 challenge creates or uses an existing ingress to create a route for Let’s Encrypt to determine that you control the domain. If you do not yet have an ingress configuration or have Ingress setup yet, I suggest checking out my article Ingress on Custom Kubernetes, which builds on the Production Hobby Cluster guide.

Create the test Certificate with kubectl:

kubectl create -f 30-Cert-DOMAIN.yml

Once we create the Certificate object we can check the status to find any errors. Generating staging certs give us opportunities to fix any mistakes and run the request for a cert multiple times without running into rate limitations.

Check the status with the following:

kubectl describe certificate example-com

Under Conditions: look for Certificate issued successfully. If the certificate issued successfully, you can view it in the Secret defined in your configuration. In our example, the secret is named example-com-staging-tls.

kubectl get secret example-com-staging-tls -o yaml

In my experience you should include the Let’s Encrypt environment in the Secret name, this avoids confusion when you update the Certificate to production, you want to see the new production secret generated.

§Production

Edit the Certificate configuration to point to the Let’s Encrypt production environment URL, server: https://acme-v01.api.letsencrypt.org/directory. Change the secret name to include the word production rather than staging and apply the updated configuration.

Apply the production changes to the Certificate configuration with kubectl:

kubectl apply -f 30-Cert-DOMAIN.yml

Once again, check the status:

kubectl describe certificate example-com

If you get Certificate issued successfully then you are now ready to use the new example-com-production-tls secret (cert) assuming you named it after your real domain.

§Using the new cert for Ingress

Now comes the easiest part, using the new Let’s Encrypt signed x509 certificate. I’ll stick with using the domains example.com and www.example.com for purposes of illustration.

Sample Ingress using example cert:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example
  labels:
    app: example
    system: test
spec:
  rules:
  - host: example.com
    http:
      paths:
      - backend:
          serviceName: "ok"
          servicePort: 5001
        path: /
  tls:
  - hosts:
    - example.com
    - www.example.com
    secretName: example-com-production-tls

§Real Life Example

The following is my Certificate and Ingress configuration for the domains phc.imti.co, phc.txn2.com and phc.txn2.net. I’ll refrain from publishing my actual TLS certificate Secret. Let’s Encrypt produces a Multi-Domain (SAN) Certificate, and as you might have noticed imti.co is hanging out at the bottom of the list; A separate TLD like this works but should probably be a separate cert, if I were concerned with having a matching common name. However, the sub-domain phc.imti.co is destined for a redirect, so it’s not necessary.

I created the Certificate configuration with the file name 00-phc-cert.yml. I set these certs up before I added Ingress rules but after I first pointed the DNS to the cluster. cert-manager uses the nginx Ingress as specified under http01: ingressClass: to temporarily create routes for Let’s Encrypt to verify the ownership of the domain.

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: phc
  namespace: default
spec:
  secretName: phc-production-tls
  issuerRef:
    name: letsencrypt-production
    kind: ClusterIssuer
  commonName: phc.txn2.net
  dnsNames:
  - phc.txn2.net
  - phc.txn2.com
  - phc.imti.co
  acme:
    config:
    - http01:
        ingressClass: nginx
      domains:
      - phc.txn2.net
      - phc.txn2.com
      - phc.imti.co

I named the Ingress configuration 10-ingress.yml and all the domains share one cert in the TLS Secret defined in secretName: phc-production-tls.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ok-phc
  labels:
    app: ok-phc
    system: test
spec:
  rules:
  - host: phc.imti.co
    http:
      paths:
      - backend:
          serviceName: "ok"
          servicePort: 5001
        path: /
  - host: phc.txn2.net
    http:
      paths:
      - backend:
          serviceName: "ok"
          servicePort: 5001
        path: /
  - host: phc.txn2.com
    http:
      paths:
      - backend:
          serviceName: "ok"
          servicePort: 5001
        path: /
  tls:
  - hosts:
    - phc.txn2.net
    - phc.txn2.com
    - phc.imti.co
    secretName: phc-production-tls

I could have easily created a separate cert for imti.co and grouped it under a separate hosts: section. However I’ll probably be redirecting it in the future so it suits my need at the moment.

§Port Forwarding / Local Development

Check out kubefwd for a simple command line utility that bulk forwards services of one or more namespaces to your local workstation.

If in a few days you find yourself setting up a cluster in Japan or Germany on Linode, and another two in Australia and France on vultr, then you may have just joined the PHC (Performance Hobby Clusters) club. Some people tinker late at night on their truck, we benchmark and test the resilience of node failures on our overseas, budget kubernetes clusters. It’s all about going big, on the cheap.