Best Practices buildpacks containers DevOps Best Practices How-tos kubernetes Platform Engineering Best Practices Tanzu Build Service tutorials

Getting Started with VMware Tanzu Build Service for Local Development

[Note: This post refers to the 0.1.0 Beta version. For information on the updated Tanzu Build Service 0.2.0 Beta, see our more recent post here.]

Effective (and enjoyable) development should feel like you are having a conversation with your computer. Apply a small change, and see the results. Did that function do what you thought it would? Any unintended consequences? Making these small, iterative changes—when you can quickly see the results—can make your code more elegant, less buggy, and more in line with the intention of any given feature.

To enable such effectiveness, many developers turn to local development paradigms. Pushing changes to an organization’s main branch kicks off heavy-handed integration tests, packaging, and in some cases, even a full deployment. But when it comes to the task of changing code, a shorter iteration cycle is needed, one in which the developer “conversation” can be synchronous and verbose.

Technologies like Docker and Kubernetes enable this fast, iterative cycle. And applications packaged in containers allow for a plethora of automation options when it comes to deploying code. In this post, we will look at how you can move faster and more effectively with VMware Tanzu Build Service inserted into such a local development cycle. Tanzu Build Service will monitor your development branch and automatically build your containers with every push. Then it will upload that container to your image registry for you to pull down and run locally.

First, you will use Docker Desktop to spin up a small Kubernetes cluster on which you will install a local copy of Tanzu Build Service. Then you will configure Tanzu Build Service to monitor a branch in GitHub. You will then integrate Tanzu Build Service with Docker Hub, which it will use as a repository to store the images Tanzu Build Service builds. With this configuration, whenever a change is pushed to your configured branch, Tanzu Build Service will pick up that change, build the new image, give it a distinct tag (as well as tag it “latest”), and push it to your registry. You’ll then be able to pull that image:tag combination and see how your change impacts things overall before pushing it to a main branch.

Let’s get started.

Note: At the time of this writing, Tanzu Build Service is currently in version 0.1.0, and the beta* was just opened to customers. This space is rapidly changing, and bugs are inevitable. The Tanzu Build Service engineering team is working diligently to enhance features and fix issues. This guide is not a solution meant for production use cases (yet). 

Prerequisites

Before you get started, you’ll need to do the following:

  • Install Docker Desktop (for Windows or Mac): You will use it to create your local Kubernetes cluster, as well as to run your containers built by Tanzu Build Service.
  • Create a Docker Hub account: This is where Tanzu Build Service will push your built images for you to pull down and run. It is also where you will push your Tanzu Build Service build images during the install process.
  • Install the Pivnet CLI: This CLI offers an easy way to download applications and bundles from the VMware Tanzu Marketplace, straight from the command line, without worrying about passing credentials in curl headers. If you would prefer you can modify commands used in this post to use curl, wget, or similar.
  • Install the Kubectl CLI: You will use this CLI to manage your Kubernetes environment. Tanzu Build Service also uses your Kubernetes config file to connect and authenticate with your local cluster. 
  • Install Duffle (either open source or from the VMware Tanzu Network): Duffle is an application for installing and managing distributed applications like Tanzu Build Service. You will use it here to push the Tanzu Build Service installation image to your registry, as well as install Tanzu Build Service onto Kubernetes. The version from the Tanzu Network is preferred for this project since it is tested with the Tanzu Build Service version deployed. 

Clone this GitHub project

To install Tanzu Build Service, you will need to create several configuration files. To make some of the commands provided here a little more user-friendly, I have added example configuration files to this repo, which we will go over throughout this post. While it is certainly possible, and relatively easy, to create all of this from scratch, those should make the process a little easier.

Clone the repo into your local working directory, and cd into it. This pwd is where we assume all of the following commands will be run from.

> git clone https://github.com/anthonyvetter/tbs-deploy.git && cd tbs-deploy

Download the Tanzu Build Service bundle

Tanzu Build Service is hosted on the VMware Tanzu Network (formerly the Pivotal Network, or PivNet). You will need access to this network as a customer to download Tanzu Build Service and the associated client tooling. This command will download Tanzu Build Service version 0.1.0 into your local directory. Download commands for other releases can be found on the VMware Tanzu Network.

> pivnet download-product-files --product-slug='build-service' --release-version='0.1.0' --product-file-id=648378

Create the Kubernetes cluster

This is where you will install Tanzu Build Service. To create a local Kubernetes cluster using Docker Desktop, follow this guide from Docker. The process is straightforward:

Docker Desktop > Preferences > Kubernetes > Check “Enable Kubernetes” > Apply & Restart

Kubernetes running on Docker Desktop provides an easy way to get started with Kubernetes, as well as a way to reset your cluster to “defaults” should something go awry. Creating this cluster will also create a Kubernetes config file in ~/.kube/config which kubectl and Tanzu Build Service will use to interact with your new cluster.

Switch your kubectl context to your local cluster.

> kubectl config use-context docker-desktop

And verify the cluster is working properly.

> kubectl cluster-info
Kubernetes master is running at https://kubernetes.docker.internal:6443
KubeDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Create the Tanzu Build Service credentials file

The credentials file tells Tanzu Build Service where your Kubernetes config file is in order to connect to it. It also defines credentials for Tanzu Build Service to connect to your image repository in order to complete the install. Since this guide uses Docker Hub for the install, we don’t need to define credentials for the registry in this step. This file can be found in the tbs-creds directory of the GitHub project. Open the file in your editor of choice, and modify the path as needed (likely just the username).

❯ cat tbs-creds/credentials.yml
name: build-service-credentials
credentials:
- name: kube_config
  source:
    path: "/User/anthonyvetter/.kube/config"
  destination:
    path: "/root/.kube/config"

If you want to use Tanzu Build Service with other registries, see the Tanzu Build Service install guide for further instructions. 

Push the Tanzu Build Service bundle to Docker Hub

The bundle you downloaded earlier contains many images for the buildpack runtime packages Tanzu Build Service will use to build your images. It also contains the install image for Tanzu Build Service. For Tanzu Build Service to have access to these images, they need to be pushed to your image registry. 

Note: Pushing them will create many separate image registries within your Docker Hub account—one for each buildpack image, as well as one for the install image. More on these in the next step. You may consider using a separate Docker Hub account for this project, as these image registries might clutter your interface.

First, log in to your Docker Hub account. This command will ask you for your username and password.

> Docker login index.docker.io/anthonyvetter -u yyyyyyyyyyyyyyyyyyyyyyyyyyy -p xxxxxxxxxxxxxxxxxxxxxxxxxxx

Then use Duffle to push the bundle to the repo. This command will take some time to complete; depending on your internet connection, it should take anywhere from 5-10 minutes. Modify this account with your own account name.

> duffle relocate -f build-service-0.1.0.tgz -m relocated.json -p index.docker.io/anthonyvetter

The -m attribute is a JSON formatted output file we will use later to tell TBS where each image is located. 

Install Tanzu Build Service 

Now it’s time to install Tanzu Build Service to your local Kubernetes cluster. This command will utilize many of the environment variables we previously defined.

In addition, you can change the install name from tbs-deploy-demo to something that makes more sense for your use case. The kubernetes_env attribute is the name of your Kubernetes cluster. If you’re following these steps the name is likely docker-desktop, but running “kubectl config get-clusters” will give you a list of your configured clusters. Then change the docker_repository attribute for your username and set the credentials.

> duffle install tbs-deploy-demo -c ./tbs-creds/credentials.yml 
--set kubernetes_env=docker-desktop
--set docker_registry=index.docker.io
--set docker_repository=“index.docker.io/anthonyvetter/”
--set registry_username=“yyyyyyyyyyyyyyyyyyyyyyyyyyy”
--set registry_password=“xxxxxxxxxxxxxxxxxxxxxxxxxxx”
 --set custom_builder_image=“index.docker.io/anthonyvetter/cf-build-service-dev-219913-invocation-image-d370867b6d9f765a72bc7ad10367ca5f:0.1.0-rc.14”
-f ./build-service-0.1.0.tgz
-m ./relocated.json

Again, this will take a couple of minutes, but not as long as it took to push the images to Docker Hub.

Note: While this guide is for setting everything up yourself, it is worth noting what the responsibilities of different roles within an organization would be if this were a production use case. Ordinarily, an operations team would be setting up and managing the Tanzu Build Service cluster environment, managing the cluster credentials and buildpack images, and generally doing all of the steps you just completed.

Everything that follows would typically be done by a developer. They would simply install and connect the Tanzu Build Service client application, pb (or more appropriately, script the install as part of setting up a developer environment), and set up the automated build flow.

Install pb

The local CLI tool, pb, is used for interacting with your Tanzu Build Service installation. It can be downloaded from the VMware Tanzu Network for your platform here. The following commands are specific to macOS. Modify them as needed for other platforms.

To download the binary for macOS, use the following pivnet command. Go to the Tanzu Build Service bundle in the VMware Tanzu Network to find versions for other platforms.

> pivnet download-product-files --product-slug='build-service' --release-version='0.1.0' --product-file-id=648384

Once downloaded, rename the binary.

> mv pb-0.1.0-darwin pb

Make the binary executable.

> sudo chmod +x pb

And move it into your local $PATH.

> mv pb /usr/local/bin

Verify the installation of your Tanzu Build Service server and pb client by running a command to list all the builders. You should have just the default builder.

> pb builder list
Cluster Builders
----------------
default

Tanzu Build Service includes the ability to create custom builders, with limited permission scope and included runtimes. More on managing and creating builders can be found in the Tanzu Build Service docs. We are using the default builder for this guide.

Create an example application

Before Tanzu Build Service can build a container image for us, we need an application for it to build. In this project, the application I am using is called node-todo. This is a simple example of a todo app that is relatively easy to understand and make changes to as you experiment. You can fork this into your own repository to follow along or configure a different image. If you do use a different image, just be sure to modify the rest of the commands in this guide to suit your needs.

Forking is done through the GitHub GUI. A guide for how to do that can be found here.

Configure a project and users

A project in Tanzu Build Service is a way of segmenting images and managing permissions. Neither are required to complete this guide, since there is only one user, but it helps to understand the concepts. And if you decide to use this install for multiple projects, you will want to set up projects and users to keep things organized.

Creating a project will also create a namespace in your Kubernetes cluster where your local image will run.

> pb project create node-todo

Next, we need to target this project. By targeting it, all subsequent pb commands will be against this project, including configuring secrets and adding users.

> pb project target node-todo

Now we can add a user to the project.

> pb project user add anthonyvetter

And now you can verify that your user and project were created successfully. You can also add groups, but that’s probably not necessary given the intent of this installation. Again, detailed information on users, projects, and groups can be found in the Tanzu Build Service documentation.

> pb project members
Project: node-todo

Users
-----
anthonyvetter
docker-for-desktop

Groups
------

Configure secrets

Tanzu Build Service will need to talk to your version control system and your image registry. In this guide, you are configuring Tanzu Build Service with GitHub and Docker Hub. To do this, you need to pass these credentials to Tanzu Build Service using the pb secrets command. Example files can be found in the secrets directory of the GitHub project.

The github-config.yml contains the name of the project you are working on, your registry address, a username, and your password (you will need to add a personal access token within GitHub for this). Later on, we will use Tanzu Build Service to build a sample application, so the project below is named after that application. Modify this file to point to your own GitHub repository, as well as your own credentials.

> cat secrets/github-config.yml
project: node-todo
repository: github.com/anthonyvetter/
username: yyyyyyyyyyyyyyyyyyyyyyyyyyy
password: xxxxxxxxxxxxxxxxxxxxxxxxxxx

The dockerhub-config.yml is similar. The project should be named the same as it was for GitHub. For the registry entry, you need to explicitly call out the API version (v1) and the protocol (https://). Modify this file to add in your own Docker Hub credentials.

> cat secrets/dockerhub-config.yml
project: node-todo
registry: https://index.docker.io/v1/
username: yyyyyyyyyyyyyyyyyyyyyyyyyyy
password: xxxxxxxxxxxxxxxxxxxxxxxxxxx

Then apply each of them to your Tanzu Build Service installation, starting with GitHub.

> pb secrets git apply -f secrets/github-config.yml

And then for the registry.

> pb secrets registry apply -f secrets/dockerhub-config.yml

These apply the secrets to the targeted project—in our case, node-todo. Once those secrets are applied, different Tanzu Build Service projects can interact with different registries and repositories, giving you the flexibility you need for this local-dev project, but at a much larger scale!

Configure an image

So now our Tanzu Build Service project knows about the two systems it needs to talk to. The last step is to tell Tanzu Build Service about the code we want to monitor and build. The sample app we are using is node-todo. The image config is located in the images directory of the GitHub project.

Here we are telling Tanzu Build Service where to retrieve the source code. Tanzu Build Service will be configured to watch the master branch, but you can configure it to watch your own development branch for whatever feature or bug you happen to be working on. Finally, the tag is where the image will be pushed in your registry. Modify this file to address your own image.

❯ cat images/node-todo-config.yml
source:
  git:
    url: https://github.com/anthonyvetter/node-todo
    revision: master
image:
  tag: index.docker.io/anthonyvetter/node-todo

Then apply the image.

> pb image apply -f images/node-todo-config.yml

Once the image configuration has been applied, Tanzu Build Service will scan the repo and start building your image. You can see this happening by running the pb command below to list all of the builds associated with an image tag.

> pb image builds index.docker.io/anthonyvetter/node-todo
Build    Status      Started Time           Finished Time    Reason    Digest
-----    ------      ------------           -------------    ------    ------
    1    BUILDING    2020-04-14 15:14:43    --               CONFIG    --

Once completed, Tanzu Build Service will put a copy of the image into your Docker Hub registry, as well as a copy on your local Kubernetes cluster within the node-todo namespace. From there, you can ‘docker run’ the image and see any changes you push to your registry.

Next, try making a change to the source code and pushing it to the branch your Tanzu Build Service project is configured to watch. Tanzu Build Service will pick up the change automatically, build a new container, and push it to your registry!

All of this applies to your base image, too. If your base image is `FROM node:13-alpine’ and a new patch release of node 13 or Alpine comes out with some critical security fixes, Tanzu Build Service will pick up those changes and rebuild and redeploy all of your containers. This is a very powerful feature of Tanzu Build Service, one that helps keep your customers, and your business, safe from vulnerabilities.

If you would like to dive deeper into VMware Tanzu Build Service, check out the documentation section. And make sure to let us know how the beta* is going!

*Note there is no commitment or obligation that beta features will become generally available.

This article may contain hyperlinks to non-VMware websites that are created and maintained by third parties who are solely responsible for the content on such websites.

Related Articles