2 approaches to deploying container as a service solution in your private datacenter using vSphere and vRealize Automation

Container technologies have become ubiquitous in the modern datacenter, their advantages for application packaging are undeniable. Developers are leading the charge, adopting container technologies at a rapid rate and demonstrating their advantages by bringing both new and updated applications to market sooner. One approach to solving these challenges is for IT to offer Containers as a Service. In doing so you, as IT, can add significant value to your company’s developer community while at the same time providing a level of standardization and governance conducive with running containers in production.

This purpose of this post is to describe and discuss two container platform solutions, the high-level architectures and assist you with making decisions about which solution or combinations of solutions will best meet your requirements.

CaaS Ecosystem


Looking from a pure container perspective CaaS includes a Docker compatible container hosts and a private container registry. These are the minimal components allowing developers to create, store and deploy their container images in a secure way. In order to provide this capability infrastructure administrators, have to perform series of tasks related to preparing container hosts, their network and storage access policies, apply governance rules to production workloads, etc. Finally, when everything is set and it is operational it is time to start monitoring the life system, determine resource usage, troubleshoot network bottlenecks, find orphan hosts and containers, and many more.

Most of these tasks can be easily addressed using some of the already deployed virtual infrastructure components defining an instance of a CaaS ecosystem which can facilitate both implementations in the Container Plane with a different level of support.

CaaS common use cases:

  • Creating new container NSX networks or vSAN datastores on demand – vRA XaaS offers a really easy way to define such self-service operation and apply proper governance on top of it
  • Deploy container hosts (and private registry) – depending on what container strategy is chosen this can be implemented as standard or XaaS vRA blueprint
  • Deploy pure container or hybrid container plus virtual machines applications – simply use Admiral container management solution, or in combination with vRA composite blueprint
  • Move and deploy container images from development space into production – implement vRCS pipeline to automate this process and apply proper governance rules
  • Monitor resources containers and container hosts – vROPs and Wavefront provide really powerful toolsets for monitoring both VIC and VM container hosts
  • Inspect network traffic and flow – NSX offers basic powerful troubleshooting tools such Traceflow and Flow Monitoring, while vRealize Network Insight brings much more on the stage
  • Troubleshooting – there are three major ways how containers do logging, through application itself, through host volume or Docker daemon driver, where all of them can be configured with a vRLI target providing cross concern analytics system

Basic CaaS using a Virtual Machine as a container host with vRA


For a basic Containers as a Service solution that is straight forward to roll out in an existing vSphere with vRealize Automation environment you can use the self service capability of vRA to deploy Virtual Machines that can host multiple containers. Two examples of operating systems that can provide this capability are Photon OS and Core OS. See this blog post for a basic overview of how to configure vRA to provide CaaS.

Chose this platform when:

  • You have an existing vRA infrastructure
  • You are looking to provide a container as a service solution to your customers in the shortest possible timeframe
  • You want to test out a container solution to determine how your developers will adopt containers in the future
  • The containers being used need to be provisioned and decommissioned quickly such as in a high churn development environment

CaaS using vSphere Integrated Containers (VIC) to host containers


VIC can provide an alternate solution to deploying large virtual machines that host many containers. The VIC Virtual Container Host solution deploys individual VMs for each container. This means that your existing monitoring solutions can provide granular statistics for each deployed container and you can use vSphere resource pools to manage the container resources across multiple ESXi hosts.

Chose this platform when:

  • You have containers that you need to put into production and use your existing production virtual machine monitoring systems to monitor individual containers
  • You want to assign an individual vNic or external IP to some or all of the containers and use your existing firewall or traffic inspection solutions
  • You want to use resource pools to allow your container workloads to be load balanced across multiple hosts
  • Isolation is also an important factor: VM-level isolation is well understood and trusted and can help simplify meeting regulatory requirements when deploying containers “as VMs” with VIC.

Other Components in Your Container Architecture

Container Registry:

When deploying a private cloud CaaS solution you should consider deploying a private container registry. The registry is the storage location for containers that have been created by your developers. Once a new container is built or an existing container has been updated it will be stored in the registry ready to be deployed into your test environment and finally into production. VMware’s Harbor is an example of a container registry that provides the controls necessary to integrate in a production-ready environment (e.g. RBAC against a directory, auditing, and replication.)

Container Management:

Deploying containers is much simpler and faster than deploying virtual machines, for this reason once you provide the capability you are likely to quickly run into a container sprawl situation. Being able to track and manage individual containers will help to reduce container sprawl. VMware’s OpenSource project Admiral, which is included with vRA, provides the container management capability and brings the additional governance capabilities of vRA to your container infrastructure.


Teodor Simchev has more than five years of experience in VMware R&D with primary focus on DevOps. Now he is member of Professional Service Center of Excellence Team delivering custom solutions with VMware products.

James Wirth works in the Professional Service Engineering Team designing services solutions for VMware customers. He is a proven cloud computing and virtualization industry veteran with over 10 years’ experience leading customers in Asia-Pacific and North America through their cloud computing journey. @jameswwirth