VMware vSphere with Tanzu Update 2 (U2) is packed with new features, among them the ability to provision new Tanzu Kubernetes clusters with any container registry certificate, including those that are private or self-signed. This feature brings critical functionality to anyone who wants to run an internal private registry or multiple registries. Without it, image pulls are met with a x509 certificate signed by unknown authority error. Alternatively, if an administrator manually installed the certificate on every Kubernetes node, a rolling update would break the configuration. vSphere with Tanzu now has the capability to automatically install self-signed certificates on Tanzu Kubernetes clusters to make pulling images a breeze. If you want to skip the reading and watch a video demo, scroll to the bottom and check it out.
Harbor is an open source registry and a graduated project in the Cloud Native Computing Foundation. It comes with multiple enterprise features, such as role-based access control, image vulnerability scanning, proxy caching for things like DockerHub rate limiting, and much more. Harbor support is included with any Tanzu Edition license and can be deployed in a few different ways. Let’s take a look to see how we can provision a Tanzu Kubernetes cluster with self-signed certificates from a Harbor registry.
Provision, authenticate, deploy
The first thing we need is a new Tanzu Kubernetes cluster where Harbor will be installed. After a vSphere Namespace has been configured and the kubectl binaries for vSphere with Tanzu have been added to your $PATH, authenticate to the vSphere Namespace and provision a Tanzu Kubernetes cluster. (To do so, refer to the simple demo manifest located below.) Setting a default StorageClass is required in order for Harbor’s Helm deployment to successfully complete. Apply this manifest to the vSphere Namespace and wait for the deployment to complete. For production environments, consider using three control plane nodes, additional workers, and larger machine classes.
apiVersion: run.tanzu.vmware.com/v1alpha1
kind: TanzuKubernetesCluster
metadata:
name: harbor
namespace: tanzu01
spec:
distribution:
version: v1.18
topology:
controlPlane:
count: 1
class: best-effort-xsmall
storageClass: k8s-storage
workers:
count: 2
class: best-effort-xsmall
storageClass: k8s-storage
settings:
storage:
defaultClass: k8s-storage
After the cluster is running, authenticate to it and set proper role bindings so containers can be created with correct permissions. In a test environment, running the following command will negate pod security privileges and allow anyone authenticated to the cluster to deploy containers: kubectl create clusterrolebinding psp:authenticated --clusterrole=psp:vmware-system-privileged --group=system:authenticated.
It’s almost time to deploy Harbor. Follow the directions for deploying Harbor with high availability via Helm to edit the values.yaml file. There are four lines that require editing for this demonstration. Another new feature in vSphere with Tanzu U2 is the ability to set static IPs with ServiceType LoadBalancer when using NSX-ALB or NSX-T. Choose any IP from the LoadBalancer pool that can be applied to Harbor. In values.yaml, edit the type to be loadBalancer; under tls -> auto -> commonName, change it to be the IP address or FQDN; under loadBalancer, change the IP to the chosen IP address; and edit the externalURL to be the IP address or FQDN. For a production environment, consider increasing the storage sizes for persistent volumes and using a highly available PostgreSQL and Redis database for Harbor.
After using Helm to deploy Harbor to the cluster, we can see that a load balancer was provisioned from NSX-ALB, and that the Harbor UI is accessible.
Get vSphere to trust the certs
The next step is editing the vSphere with Tanzu configuration to trust the self-signed Harbor certificates. Getting the certificate is fairly straightforward. Use openssl s_client -connect IPorFQDN:443 and copy the portion where it shows ----BEGIN all the way to -----END CERTIFICATE-----.
Now we need to Base64 encode this certificate value onto a single line. I’ve discovered the best way to do this is using the Text to Base64 converter on Base64.guru.
Copy the Base64-encoded text and change the kubectl config to use the context of the namespace. Edit the running configuration with kubectl edit TkgServiceConfiguration and add the new trust fields, which will configure any new Tanzu Kubernetes cluster to automatically trust this certificate. Then simply copy and paste the external private registry configuration from the documentation. Using :wq will save the running configuration.
Push a test image
Now it’s time to test it out. Deploy a new Tanzu Kubernetes cluster, and once that’s under way, go to Harbor and create a new public project called Tanzu. Use your local Docker installation to pull the latest NGINX image from DockerHub with docker pull nginx. Tag it with your Harbor installation, for example docker tag nginx:latest IPorFQDN/tanzu/nginx:v1. For this next bit, your local Docker daemon needs to trust your Harbor registry. You can facilitate that trust in the Docker Engine configuration by adding the Harbor IP or FQDN.
Once the local Docker daemon trusts the insecure registry, log in with docker login IPorFQDN and push the image with docker push IPorFQDN/tanzu/nginx:v1.
Test it out
After the new Tanzu Kubernetes cluster has been deployed, and pod security privileges have been negated using the previous command performed on the Harbor cluster, it’s time to test it out. Use kubectl create deployment --image IPorFQDN/tanzu/nginx:v1 tanzu-nginx; it should pull the image successfully!
We can also take this a step further, to expose the deployment with a service. It will pull an IP from the LoadBalancer pool and NGINX will be accessible from the browser.
This new feature brings a lot of extensibility to vSphere with Tanzu because new Tanzu Kubernetes clusters can pull images from any private registry. Take a few minutes to watch the demo video to see the entire process.
Learn more about using an external container registry with Tanzu Kubernetes clusters and check out the free courses from ModernApps.ninja to try more vSphere with Tanzu features.
