One of the new features that came with the latest update of vSphere with Tanzu was the ability to use TKG Extensions. This powerful framework allows simplified deployment and management of multiple open source projects that are backed by VMware support, including alerting with Prometheus, visualization with Grafana, ingress with Contour, and logging with Fluent Bit.
This post will walk through the setup and configuration of Fluent Bit using TKG Extensions on a Tanzu Kubernetes cluster. After the setup is complete, all the logs from the cluster will be forwarded to vRealize Log Insight using syslog, which can be used for log retention and analysis. Fluent Bit has the ability to forward logs to multiple collectors, such as Elasticsearch, HTTP, Kafka, and Splunk, so it’s highly configurable.
The first step is making sure there is a target to collect logs. Deploying the VMware vRealize Log Insight Virtual Appliance is straightforward, or sign up for the VMware vRealize Log Insight Cloud. Once up, it’s ready to start collecting!
TKG Extensions require three prerequisite components—Extension Manager, Kapp Controller, and Cert Manager—each of which are located in the TKG Extensions bundle at /tkg-extensions/extensions/. It may appear that there are duplicate folders, so to ensure successful deployment, check that you’re using the extensions folder. Then install the components and verify that each of them is running.
cd /tkg-extensions/extensions/
kubectl apply -f tmc-extension-manager.yaml
kubectl apply -f kapp-controller.yaml
cd /tkg-extensions/cert-manager/
kubectl apply -f cert-manager/
Deploying Fluent Bit requires three easy steps. First, create the namespace and role objects. This establishes a home for where the Fluent Bit containers will run and ensures that several role-based access control objects are used.
cd /tkg-extensions/extensions/logging/fluent-bit/
kubectl apply -f namespace-role.yaml
The second step is to create a copy of a Fluent Bit configuration file, editing it with the values for the environment and applying the file as a Kubernetes secret. vRealize Log Insight, a syslog collector that listens on tcp/514, can use multiple RFC formats. The example below uses RFC 5424.
cp syslog/fluent-bit-data-values.yaml.example syslog/fluent-bit-data-values.yaml
vi syslog/fluent-bit-data-values.yaml
kubectl create secret generic fluent-bit-data-values --from-file=values.yaml=syslog/fluent-bit-data-values.yaml -n tanzu-system-logging
After the secret has been applied, the final step is deploying the Fluent Bit extension.
kubectl apply -f fluent-bit-extension.yaml
Shortly after the deployment, logs will start appearing in vRealize Log Insight.
To examine what happens with an individual application, let’s create a simple NGINX deployment:
kubectl create deployment --image nginx my-nginx
In vRealize Log Insight, go to Interactive Analytics and perform a search for nginx to bring up all the logs needed to create the pod.
TKG Extensions provide an easy framework for getting started with some of the most popular open source applications in the Cloud Native Computing Foundation landscape. In the future, the framework will also include simplified lifecycle management of these existing extensions, and will package even more extensions—all while being supported by VMware.
Make sure to read the documentation for the latest instructions on how to install and configure TKG Extensions. For more, check out the free containers, Kubernetes, and Spring on-demand labs and Tanzu products training.
Image courtesy of Billy Hyunh via Unsplash.
