VMware Cloud Foundation Sofware-Defined Data Center

Deep dive into VMware Cloud Foundation – Part 1 Building a Nested Lab

If you’re like me, you like to dive deep into technology to learn how things work in the safety of a lab environment. I enjoy experimenting with a live lab. I’ll use the lab to look behind the curtain and see what’s going on behind the scenes. The safety of the lab allows me to try every button and configuration option without the worry of breaking production infrastructure systems. I have decided to do a deep dive blog series on Cloud Foundation, and take some time to introduce all the inner workings of VCF. To facilitate this journey, anyone following along needs a working lab environment. To begin, I want to introduce you to an internally developed tool called VMware Cloud Foundation Lab Constructor (VLC).

Part 1 – Introducing the VLC

VLC is an automated tool that deploys an entire nested Cloud Foundation environment onto a single physical host or vSphere Cluster.

Introduction to Cloud Foundation

Before we begin with describing the VLC, we need first to understand what Cloud Foundation is and what the requirements are for installing it.

  • Software Components

VMware Cloud Foundation is a fully integrated software platform powered by the core components of the Software-Defined Data Center (SDDC). Products included during an initial greenfield implementation are vSphere, vSAN, NSX, SDDC Manager & vRealize Log Insight.

  • Physical  Production deployment requirements

To deploy Cloud Foundation onto bare metal infrastructure requires four physical vSAN Ready nodes, a minimum of (2) 10GB switches, and an upstream BGP router.  These hosts must then be configured with a specific installation of ESXi so that they are ready for use in Cloud Foundation. This minimum specification of four hosts provides capacity for what is called the Cloud Foundation Management Domain. Depending on customer needs, Cloud Foundation can scale up to include more hosts to deploy additional applications. Adding more hosts later also allows the end-user to create Workload Domains.

  • VCF XLSX or JSON Configuration file

Installing the fully integrated stack of software components for Cloud Foundation requires a Microsoft Excel Spreadsheet containing parameters necessary to deploy Cloud Foundation. This spreadsheet can optionally be converted to JSON file before VCF deployment. The parameters included in the JSON are, DNS names, IP and subnet information, passwords, NIC selection, Upstream BGP neighbor, license keys. Along with details for deploying vSphere, vSAN, and NSX configurations. In short, this file contains all the details necessary to deploy Cloud Foundation. Throughout this blog series, I will refer to this JSON configuration file and demonstrate how the VLC leverages it to dynamically build nested virtual infrastructure.

  • VCF Deployment Process

A Cloud Builder appliance automates the deployment of Cloud Foundation. Cloud Builder is a purpose-built OVA that is capable of deploying the Cloud Foundation Management Domain on these first four hosts. It’s important to note when deploying Cloud Foundation to physical hardware; this process can take approximately three hours to deploy the Management Domain. Now you might be thinking, this is a long amount of time. In reality, this is extremely fast when you understand that VCF is deploying vSphere, vSAN, NSX, and vRealize Log Insight, all as a highly integrated solution.

VLC – A magical lab in a small form factor

VLC has been developed as an unsupported lab tool that can begin with a single physical host, loaded with ESXi.  Note: Adding physical hosts later allows you to expand, but this one host provides the bare minimum resources needed to begin the installation. Nested within this single host, VLC virtualizes all the hardware requirements mentioned above to install VMware Cloud Foundation. By nesting the Cloud Foundation hardware using VLC, the lab resource requirements become drastically reduced. Allowing as many people as possible to follow along with this blog and build a lab environment of your own. Achieving the reduction of required physical resources happens in two stages.

In the first stage, VLC leverages ESXi transparent page sharing, the ESXi CPU scheduler, and thin disk.

The second stage of resource reduction comes from specifying that Cloud Foundation deploys the Core SDDC components in the smallest possible form factor. Specifically, components like the vCenter and vRealize Log Insight nodes are deployed in the tiny, and xsmall format as specified in a JSON config file. With these two stages, the reduction of physical resources needed to deploy the VLC nested lab components becomes possible on a single physical host with 12 CPU Cores, 128 GB RAM, and 2 TB of SSD Disk.

Architecture Layers

Using nested virtualization for the lab allows VLC to automate all prerequisite components for deploying the lab. Let’s start by explaining how the VLC is virtualizing a full hardware set for Cloud Foundation. Below is a physical to logical view of the components used in VLC.

  • Creating Layer 1

Layer 1 is the end users physical lab equipment. This can be one host or multiple hosts set up in a cluster with a vCenter. Providing the physical equipment to run on is up to the end-user.

  • Creating Layer 2

In Layer 2, VLC creates four new nested ESXi hosts using the ESXi API via PowerShell CLI. These nested hosts are then bootstrapped and loaded with a custom-built ESXi image.

  • Deploy Cloud Builder

VLC will then deploy the Cloud Builder appliance onto Physical (Layer 1) host and attach it to the port group configured for VLC.

  • Automating Requires Services

Deploying Cloud Foundation requires properly configuring the following external services, DNS, NTP, DHCP, and BGP routing.  One feature of VLC is injecting these required services into the Cloud Builder appliance. This simplifies the lab deployment process, and by using automation to configure them eliminates potential typos and provides a very fast and smooth lab deployment process.

  • Creating Layer 3

Layer 3 is the deployment of Cloud Foundation onto the nested hosts created by VLC in Layer 2. VLC uses the JSON file necessary for deploying Cloud Foundation, and if selected, executes the Cloud Builder bring-up API. When the VLC is done you have a complete working nested VMware Cloud Foundation Lab.


This concludes Part 1, an introduction to VLC for building a nested Cloud Foundation Lab. As you can see, we have come up with a clever way to use nested ESXi to build a Cloud Foundation lab automatically. Keep in mind, this is only for a lab environment, this is only for testing, and learning VCF. In my next blog, I will walk through the process of using VLC and how to get Cloud Foundation Software bits. However, if you want to dive straight into learning more about VLC and deploying a lab, you can start today. Click this link to download VLC and related documentation. http://tiny.cc/getVLC

Then Join the VLC community on Slack https://tiny.cc/getVLCSlack


8 comments have been added so far

  1. I like the idea of creating the nested VCF as a home lab.
    Any suggestions on the cheapest physical kit that would be suitable to achieve this?

  2. Heath and Team… Kudos on a well designed deployment platform! Spun this up in my home lab and can’t wait to expand upon what the platform provides!

  3. I have a Powerful Desktop PC with 64GB RAM, an AMD 3900x CPU and 2x onboard (unused) PCIe4.0 NVMe drives, and I’d like to deploy a nested VCF Lab onto it by means of Workstation, even if only for the Cloud Builder Appliance and 4 ESXi hosts.

    Would this be possible in your opinion?

      1. Maybe add an option to remove reservations so we can still deploy with less resources, maybe at a lesser experience?

  4. Excellent blog I have deployed this in my home lab. I found having 256GB Ram not the reommnded minuim of 128GB made it run so much smoother

Leave a Reply

Your email address will not be published. Required fields are marked *