My most popular series on this blog is the HOL Three-Tier App build.
Originally posted in 2017, I created an update in 2020 based on feedback from some readers that they wanted to use a newer version of Photon OS in the app. That post requires following steps in two different sets of articles and is confusing.
This new series aims to cover the end-to-end creation of an updated version of the application using Photon OS 4.0 Rev2. Some things have changed with the configuration and I have learned a bit since 2017, but the application itself remains simple: usable on its own or as a foundation for your own enhancements.
The posts in this series are here:
This simple multi-tier application has been used in some of the VMware Hands-on Labs to demonstrate network connectivity, micro-segmentation and load balancing. Since the original release, I have received a lot of feedback. People use it for demonstrations, education purposes in their home labs, POCs and troubleshooting environments. I tossed it into Docker containers to see what that looked like and one user created an Ansible playbook to deploy it:
I just wanted to let you know I created an Ansible playbook on deploying the HOL 3 Tier App and I used some of the formattings from your Github repo… Stands up in about 10 mins and only works on Photon 3 atm.
@KarlNewick via Twitter
If you use this app, please shoot me a note here or on Twitter (@dobaer) as I love hearing where people have taken this and how it can be improved while preserving its overall simplicity.
To get started, we create a base template virtual machine that is used as a foundation for the entire solution.
Build a Photon OS Template
This post steps through downloading the base operating system and performing the configuration tasks common to all VMs in the solution. As with anything, there are multiple ways to do this. This represents the way that has worked for me consistently, with a specific version of Photon OS — in this case, 4.0 Rev2.
The Need
Whether you live in a cutting-edge, microservices-oriented world, or have a traditional application spread across multiple machines, the components (machines, containers, services, processes, etc.) need to communicate with one another over the network. Understanding what that looks like is important to securing the connection end-to-end. This simple application is intended to provide a starting point for learning or testing firewall and load balancing configurations to see how they affect a distributed environment. It can also be used to test or showcase applications that monitor network traffic to map out application dependencies.
For instruction purposes, we wanted three simple, independent parts that could be deployed, rearranged, and manipulated to illustrate many different situations that may occur in an environment. Rather than being complex, it is intended to require communication among the parts on specific TCP ports in order to function. When communication is not possible, the application breaks. It is good at that.
The Application
This application is composed of three operating system instances, independent VMs, each of which handles a specific task. When all of them can communicate over the network via the required ports, the client receives the requested information and can interact with that information. If there is a breakdown, not so much.
This demonstration application has been created so that each component VM is stateless and independent from the others: IP addresses can be changed and multiple instances of the web and application tier VMs can be created by cloning, renaming, and re-addressing. The basic build with one of each type and all resources on the same subnet will be described in this series.
The diagram above illustrates what I will be covering in this series. I put SSL in here because it is always a good idea to secure your web traffic and it provides the opportunity to configure a load balancer in front of the web tier in a more realistic scenario.
Build the Base
This application was created using VMware’s Photon OS. If you are not familiar with this operating system, you can read more on VMware’s Project Photon OS page.
As the page indicates, Photon OS is a Minimal Linux Container Host. Because we have very basic needs, we are going to focus on the first half (Minimal Linux) and ignore the second half (Container Host) for now. One cool thing about Photon OS is that it boots very quickly.
Before we do anything, I’d like to give you an idea of the time involved in building this application from scratch. Once I have the software downloaded and have staged the base template, I can get the basic application up, running, and captured within 30 minutes. If you are comfortable using the vi editor and an SSH connection, I think you can as well. Even if you are a bit rusty, it should not take too much longer than an hour.
TIP: keep in mind that Python is very sensitive about spacing
Download the Software
This application runs as a set of virtual machines on my VMware ESXi hosts or within VMware Fusion/Workstation, though you can really host the application anywhere if you put your mind to it.
I selected the Photon OS 4.0 Revision 2 Release — OVA with virtual hardware v11 as my starting point.
At the time of this writing, that file was available using the link at the bottom of the Photon OS GitHub repository page. The file I downloaded was named photon-ova-4.0-c001795b80.ova and is less than 300 MB.
Import the OVA
Once you have downloaded the software, import the OVA to your environment and power it up.
Create a Baseline VM
You can handle this however you like, but I have some tasks that are common across the VMs and don’t like to duplicate work. Note that you will need Internet access from the VM to install software. You will also need three IP addresses that you can statically assign to the VMs that you will be creating (a fourth if you want to give the template its own address).
Set the root password
The default password on the OVA is changeme — use this to log in with the username root. The system will prompt you again for the changeme password and then require you to set a complex password. It didn’t like our standard (simple) password, so I had to set it to VMware123! and then I used passwd to change it to VMware1! (the password that we use in all VMware Hands-on Labs).
Note that passwd will complain about a weak password, but still makes the change to whatever you like if you are persistent:
NOTE: My convention in this document is to set examples in a monospaced font and prefix with a “#” because they are executed as the root user. You don’t type the “#” if you want the command to execute
Ensure that root’s password does not expire
It is always a drag when you finally get back to working on your lab, only to have to reset passwords… or, worse, figure out how to break in because the password you set at install time is no longer valid. In production, I would not do this, but this is a lab tool.
# chage -M -1 root
Note that is a NEGATIVE ONE after the -M in the above command!
Set the hostname
Change the hostname from the default generated name to what you want to use. For the template, I do this so that I know I have completed the work described in this section. If you’re not familiar with the vi editor, look here for a “cheat sheet” from MIT.
# vi /etc/hostname
Replace the current hostname with the new one and save, close the file.
Set a static IP (change from default DHCP)
In the OVA, the default network configuration is stored in /etc/systemd/network/99-dhcp-en.network. To configure a static IP address on the eth0 interface, rename the file and replace the contents:
# mv /etc/systemd/network/99-dhcp-en.network /etc/systemd/network/99-static-en.network
Renaming the file instead of copying it retains the permissions so that it will work and eliminates confusion about which file is used to configure the NIC. The contents required for static IPs are not very complicated. The following example is for the web-01a machine in my environment. Substitute with addresses that make sense for you.
TIP: Unless you make special concessions, don’t count on DNS to work once these VMs are deployed in DMZs or microsegments.
I configure DNS in the template because I need to be able to resolve repository hostnames to install software during the build. As for the corp.local domain, that is what we use in our labs. Feel free to replace it with your domain as long as you are consistent:
[Match] Name=e* [Network] Address=192.168.120.30/24 Gateway=192.168.120.1 DNS=8.8.8.8 Domains=corp.local
Restart the network to apply the settings, verify
# systemctl restart systemd-networkd
# ip addr show
Edit the hosts file
Because this application is intended to be self-contained, we use local hosts files for name resolution. Configuring this template with all the names and IPs that you want to use is easier than doing it later for each VM.
Specifying names here allows the other tools’ configurations to be built using names instead of IP addresses and it makes changing addresses later much easier.
Remember to also change the hostname on the loopback (127.0.0.1) from the default to your host’s name, too. This is an example of the edited file from our web-01a machine:
# Begin /etc/hosts (network card version) ::1 localhost ipv6-localhost ipv6-loopback 127.0.0.1 localhost.localdomain 127.0.0.1 localhost 127.0.0.1 web-01a # End /etc/hosts (network card version) 192.168.120.10 db-01a.corp.local db-01a 192.168.120.20 app-01a.corp.local app-01a 192.168.120.30 web-01a.corp.local web-01a
Modify the firewall to allow the desired ports
The iptables config script run at startup of the Photon OS is /etc/systemd/scripts/ip4save and only allows inbound SSH. We need to add some new rules to enable our application and pinging all around.
The new rules go in this file, below the row that looks like this:
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
New Rules:
-A INPUT -p icmp -j ACCEPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT -A INPUT -p tcp -m tcp --dport 3306 -j ACCEPT -A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT
On the template, we open ports needed for all app layers. This helps with troubleshooting during the build. If you like, you can comment out the ones you don’t need for each VM later. The main idea of this app is to implement restrictions at the network rather than on each host.
The critical inbound ports are 443/tcp for web-01a, 3306/tcp for db-01a, and 8443/tcp for app-01a.
Restart iptables to apply the new rules
# systemctl restart iptables
(optional) Verify the new rules
# iptables -L
(optional) Enable key-based SSH
If you have an SSH key that you use, now is a good time to copy your SSH key to the /root/.ssh/authorized_keys file, replacing the <ssh-key-here> text that is there by default.
(optional) Install software used by all
The OVA contains a minimal installation of Photon OS, but I created this application with the default packages in mind. We use the tdnf tool to perform installations on Photon OS. While adding lsof is optional, I find it excellent for troubleshooting.
# tdnf install lsof
Once installed, try this to see which services are listening and connected on which ports:
# lsof -i -P -n
Cool, right?
If you have anything else that you want to install — say you prefer nano to vim as a text editor — go ahead and install that now using the same tdnf syntax:
# tdnf install nano
Finish Up
I usually reboot here just to make sure everything comes up as expected before moving on.
With Photon OS, that reboot only takes a few seconds.
If everything looks good, shut this machine down and clone it to a vSphere template for use when creating the web, app, and database server machines. For this example, I named mine photon:
In the next post, I will cover the build of the database VM for our application, using this template as a starting point.
Next post: 3-Tier Demo App – 2023 Edition – db-01a
