Home > Blogs > VMware Hands-On Lab (HOL) Blog > Tag Archives: network

Tag Archives: network

HOL Three-Tier Application, Part 5 – Use Cases

If you have been following along in this series, first of all, thank you! Here is a summary of our work so far:

Next, you should have a basic three-tier application created:


A Simple Three-Tier Application

I tried to use simple components to make it usable in either a home lab or a nested environment, so they should perform exceedingly well in a real environment.

Virtual Machine Profile

The component Photon OS machines boot in a few seconds, even in our nested environment, and their profiles are fairly conservative:

  • 1 vCPU
  • 2 GB RAM
  • 15.625 GB disk

Once configured as indicated in this series, these VMs will export as OVAs that are around 300 MB each, making them reasonably portable.

The storage consumed after thin-provisioned deployment is less than 650 MB for each virtual machine. At runtime, each consumes an additional 2 GB for the swapfile. During boot, in my environment, each VM’s CPU usage is a little over 600 MHz and the active RAM reports 125 MB, but those normalize quickly to nearly 0 MHz and 20 MB active RAM (+23 MB virtualization overhead). You may be able to reduce their RAM allocations, but I have not tried this.

So, what can I do with this thing?

It is nice to have tools, but without a reason to use them, they’re not that much fun. We use tools like this in our labs to demonstrate various functionality of our products and help our users understand how they work. Here are a few ideas, just to get you thinking:

vMotion, Storage vMotion, SRM Protection and Recovery

The virtual machines that you created can be used as a set, but the base Photon OS template also makes a great single VM for demonstrating vMotion or Site Recovery Manager (SRM) recovery in a lab environment. They are small, but they have some “big VM” characteristics:

  • The VMware Tools provide appropriate information up to vCenter
  • They respond properly to Guest OS restart and power off actions
  • Photon OS handles Guest Customization properly, so you can have the IP address changed during template deployment and SRM recovery.
  • You can ping and SSH into them
  • You can use them to generate load on your hosts and demonstrate Distributed Resource Scheduler (DRS) functionality


We use a previous version of this application in several of our NSX labs that debuted at VMworld 2016. For a good micro-segmentation use case, you can look at HOL-1703-USE-2 – VMware NSX: Distributed Firewall with Micro-Segmentation. The manual is available for download here, or you can take the lab here.

For a more complicated use case using a similar application to demonstrate SRM and NSX integration, look at HOL-1725-USE-2 – VMware NSX Multi-Site DR with SRM. For that lab, the manual is available here and the lab is here.

Each of the tiers must communicate with the others using specific ports

  • Client to Web = 443/tcp
  • Web to App = 8443/tcp
  • App to DB = 80/tcp

You can use this application to test firewall rules or other network restrictions that you are planning to implement. If a restriction breaks the application, you can determine where and why, then try again. If you want to change the port numbers to match your needs, you can do that as well. Keeping the application simple means that modifications should also be simple.

Load Balancing (Distribution)

The basic idea here is that you can create clones of the web-01a machine as many times as you like and pool them behind a load balancer. In your lab, if you have it, you may want to use NSX as a load balancer. If you want to do that, I suggest checking out Module 3 – Edge Services Gateway in the HOL-1703-SDC-1 – VMware NSX: Introduction and Feature Tour lab, which covers how to set that up. The manual is here and the lab is here.

If you want to use another vendor’s solution, feel free to do that as well. This application is REALLY simple. Some free load balancing solutions can be implemented using nginx or haproxy. Fortunately, we already know about nginx from the build of our web servers, so I will cover that later in this post. First, though, I want to cover a DNS round robin configuration since understanding that makes the nginx load balancing simpler for the lab.

Example 1 – Load Distribution via DNS Round Robin

If you don’t have the resources for another VM, you can implement simple load distribution via DNS round robin as long as you understand a few limitations:

  1. You must have access to change DNS for your lab environment.
  2. Using only DNS, you get load distribution but not really balancing; there is no awareness of the load on any particular node. Rather, you simply get the next one in the list.
  3. There is no awareness of the availability of any node in the pool. DNS simply provides the next address, whether it is responding or not.
  4. Connecting from a single client may not show balancing since optimizations in modern web browsers may keep existing sockets open.

In this first example, I have 3 web servers (web-01a, web-02a, web-03a) with IP addresses, 31, and 32. My SSL certificate contains the name webapp.corp.local and it is loaded onto each of the web servers. The picture looks something like this:


Create the VMs

To create web-02a and web-03a, I simply clone my web-01a VM then reset the hostnames and IP addresses of each clone to the new values:

  • web-02a –
  • web-03a –

Alternatively, I can make a template from the web-01a VM and deploy the copies using Guest Customization to reconfigure them. Just make sure to populate the /etc/hosts file on the customized machines since the process wipes out and rebuilds that file.

Configure DNS

The required DNS changes are not complicated. You basically assign the name webapp.corp.local to the IP addresses of your web servers and set the time-to-live (TTL) to a low, non-zero value.

Using PowerShell against my lab DNS server called controlcenter.corp.local that manages the corp.local zone, I add DNS records with a 1 second TTL, associating all of the web server IP addresses to the name webapp.corp.local:

$ttl = New-TimeSpan -Seconds 1

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webapp' -IPv4Address '' -TimeToLive $ttl

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webapp' -IPv4Address '' -TimeToLive $ttl

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webapp' -IPv4Address '' -TimeToLive $ttl

If you use a BIND DNS server, just create multiple A records pointing to the same name. BIND 4.9 or higher will automatically rotate through the records. In my case, I have a Windows 2012 DNS server, and it cycles through the addresses when the webapp.corp.local name is requested.

Testing the Rotation

Here is a simple example of what this looks like from an ESXi host in my lab. A simple ping test shows the rotation occurring as intended:

[root@esx-03a:~] ping -c 1 webapp.corp.local
PING webapp.corp.local ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=1.105 ms

--- webapp.corp.local ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.105/1.105/1.105 ms

[root@esx-03a:~] ping -c 1 webapp.corp.local
PING webapp.corp.local ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=1.142 ms

--- webapp.corp.local ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.142/1.142/1.142 ms

[root@esx-03a:~] ping -c 1 webapp.corp.local
PING webapp.corp.local ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=1.083 ms

--- webapp.corp.local ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 1.083/1.083/1.083 ms

Accessing the Application

Use the https://webapp.corp.local/cgi-bin/app.py URL from your web browser to access the application. Within the three-tier application, the script on the app server displays which web server made the call to the application.


The script will show the IP address of the calling web server unless it knows the name you would like it to display instead. You provide a mapping of the IPs to the names you would like displayed at the top of the app.py script on the app server:

webservers = {

Simply follow the syntax and replace or add the values which are appropriate for your environment.

A Challenge Showing Load Distribution from a Single Host

Hmm… while the ping test shows that DNS is doing what we want, clicking the Refresh button in your web browser may not be switching to a different web server as you expect.

A refresh does not necessarily trigger a new connection and DNS lookup, even if the TTL has expired. Modern web browsers implement optimizations that will keep an existing connection open because odds are good that you will want to request more data from the same site. If a connection is already open, the browser will continue to use that, even if the DNS TTL has expired. This means that you will not connect to a different web server.

You can wait for the idle sockets to time out or force the sockets closed and clear the web browser’s internal DNS cache before refreshing the web page, but that is not really convenient to do every time you want to demonstrate the distribution functionality. If you want to be able to click Refresh and immediately see that you have connected to a different web server in the pool, you can use NSX or a third-party load balancer. If you want to use the tools that we have currently available, the next example works around this issue.

Example 2 – Implementing a (Really) Basic Load Balancer

Making a small change to the nginx configuration on one of the web server machines and adjusting DNS can provide a simple demonstration load balancer for your lab. This requires a slight deviation from our current architecture to inject the load balancer VM in front of the web server pool:


Three-Tier Application with Load Balancer

Note that there are better, more feature-rich ways to do this, but we are going for quick and simple in the lab.

Create the Load Balancer

Create the load balancer VM. You can deploy a new one from a Photon OS base template and go through the configuration from there, but conveniently, the difference between the load balancer configuration and that of our web servers is just one line!

So, make a copy of the web-01a VM and update its address and hostname:

  • lb-01a –

Change the nginx Configuration

On the lb-01a VM, edit the /etc/nginx/nginx.conf file

# vi +130 /etc/nginx/nginx.conf

Change line 130 from

proxy_pass https://app-01a.corp.local:8443/;


proxy_pass https://webpool.corp.local/;

This will allow us to leverage DNS round-robin to rotate through the list of web servers and distribute the load. Nginx has advanced configurations to handle load balancing, but this will get the job done for a lab or demonstration. Terminating SSL on the load balancer while using plain HTTP on the web servers allows a lot more flexibility, but the configuration changes are beyond the scope of what I want to do here.

Restart nginx

# systemctl restart nginx

Adjust DNS

Finally, adjust DNS to move the webapp.corp.local name to point at the load balancer and put the web servers into webpool.corp.local instead.

If you are using Windows DNS, you can use PowerShell. For BIND, edit and create the records as needed.

  1. Remove the existing webapp.corp.local pool by deleting all of the A records that point to the individual web servers:
$rec = Get-DnsServerResourceRecord -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -Name 'webapp' -RRType A
if( $rec ) { 
  $rec | % { Remove-DnsServerResourceRecord -InputObject $_ -ZoneName 'corp.local' -Force }

2. Create a new webapp.corp.local A record that points to the lb-01a machine:

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webapp' -IPv4Address ''

3. Create the new webpool.corp.local that contains the individual web servers:

$ttl = New-TimeSpan -Seconds 1

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webpool' -IPv4Address '' -TimeToLive $ttl

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webpool' -IPv4Address '' -TimeToLive $ttl

Add-DnsServerResourceRecordA -ComputerName 'controlcenter.corp.local' -ZoneName 'corp.local' -name 'webpool' -IPv4Address '' -TimeToLive $ttl

Access the Application

Now, point your web browser to the https://webapp.corp.local/cgi-bin/app.py URL. Each time you click Refresh in your web browser or enter a new search string in the Name Filter box and click the Apply button, the data refresh and the Accessed via: line should update with a different web server from the pool:


Rotating through web servers in the pool

Because the web browser’s connection is to the load balancer VM, which controls which web server receives the request, we eliminate the issue experienced when using only DNS round robin. This very basic implementation does not handle failed servers in the pool and is not something that would be used in production, but, hey, this is a lab!

It is possible to extend this idea to put a load balancer in front of a pool of application servers as well: replace line 130 in each web server’s /etc/nginx/nginx.conf file with the URL of an app server pool instead of pointing them directly at the app-01a VM.

That’s a Wrap!

That concludes the series on building a minimal three-tier application. I am hopeful that you have found this interesting and can use these tools in your own environment.

Thank you for reading!

HOL Three-Tier Application, Part 4 – Web Server

This is the fourth post in the series about building a three-tier application for demonstration, lab, and education purposes. If you have been following along, you have created the base Photon template as well as simple database and application servers.

This post covers the final layer of our stack, the web presentation tier. I have said it before, but the configuration of the web server here is really simple, and if you have made it this far, you’re golden.

The Web Server (web-01a)

All of the hosts in this application run “web server” software, but this one has the web server designation because it is the one that the user directly accesses. The entire back end could be replaced with real application middleware and an RDBMs, but the user expects this one to present an SSL-encrypted web page on port 443. This time, I have chosen not to use Apache, and there is no need for Python. There is no CGI, and minimal configuration is required aside from issuing another certificate for the SSL. This tier is mostly interesting because it will support the virtual name of the application in addition to the real name(s) of your web server(s).

The red box in the following diagram highlights the component that we are building in this post.


Again, the first steps look quite a bit like the steps we performed for the application and database servers as we assign the personality to the template. I will again outline the steps here as a reminder. Details can be found in my post about the database server.

Let’s get started!

  1. Deploy a copy of the base Photon template you created by following the steps in my first post.
  2. Name it something that makes sense to you for your web server. I called mine web-01a
  3. Power it up and log in as the root user
  4. Change the hostname in /etc/hostname and in /etc/hosts
  5. Change the IP address in /etc/systemd/network/10-static-eth0.network
  6. Use a SSH client to access the machine as root (makes pasting possible)

Instead of installing Apache again, we are going to use nginx. You can do the same thing with Apache, but I wanted to try something a little more lightweight and the nginx configuration for this use case is really simple.

Install nginx to be used as a reverse proxy

The web server machine will function as a reverse proxy, sending user requests bound for port 443 on this server to the application server at https://app-01a.corp.local:8443

# tdnf install nginx

The nginx install is less than 6 MB and takes a few seconds:


Configure the reverse proxy

Edit the configuration file, /etc/nginx/nginx.conf

# vi +116 /etc/nginx/nginx.conf

by adding the following at the bottom of the file, at line 116, just before the closing “}” in the file.

   # HTTPS server
   server {
      listen 443;
      server_name webapp.corp.local;

      ssl on;
      ssl_certificate     /etc/nginx/ssl/webapp.pem;
      ssl_certificate_key /etc/nginx/ssl/webapp.key;

      ssl_session_cache shared:SSL:1m;
      ssl_session_timeout 2m;

      location / {
         proxy_pass https://app-01a.corp.local:8443/;
         proxy_set_header Host $host;
         proxy_redirect http:// https://;

Notice that we need an SSL certificate and a key to make this work. We have done this before, so let’s create those now.

Make the ssl directory and switch into it

Let’s just create the certificates in the place that the server expects them to be.

# mkdir -p /etc/nginx/ssl
# cd /etc/nginx/ssl

Build the configuration file

It is common for multiple web servers to be configured in a pool, behind a load balancer. I create the certificate here using a name, webapp.corp.local. This name can be assigned to the load balancer’s VIP. If there is only one web server, as in my example here, this name can also be an alias that resolves to the one web server. For simplicity, and possibly for other use cases, the certificate configuration we build here includes the names of three web servers: web-01a, web-02a and web-03a.

Create the file webapp.conf

# vi webapp.conf

with the following contents, modified as needed for your environment:

distinguished_name = req_distinguished_name
x509_extensions = v3_req
default_md = sha256
prompt = no
C = US
ST = California
L = Palo Alto
O = VMware
OU = Hands-on Labs
CN = webapp.corp.local
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
DNS.1 = webapp.corp.local
DNS.2 = web-01a.corp.local
DNS.3 = web-02a.corp.local
DNS.4 = web-03a.corp.local

Save and close the file.

Generate the key and certificate

Note that this is a long command and you may need to scroll to the right to get all of it. Ensure it ends with “webapp.conf”

# openssl req -x509 -nodes -days 1825 -newkey rsa:2048 -keyout webapp.key -out webapp.pem -config webapp.conf

(optional) Validate that the PEM file “looks right”

I put this command here for those who want to look at the certificate. It is a good command to know in case you have a certificate file and want to know what information it contains. That can help you match the certificate to the proper host without needing to install it and then find out it is not the right one.

# openssl x509 -in webapp.pem -noout -text

Start the nginx server and configure it to startup when the VM boots

# systemctl start nginx
# systemctl enable nginx


With the other components (db-01a, app-01a) online, reachable and tested, you can test the whole solution with curl from the console of the web server

# curl -k https://web-01a/cgi-bin/app.py

This should return the data from the database in HTML format by executing the script on the application server.

You can filter the results by appending a querystring. Try this one:

# curl -k https://web-01a/cgi-bin/app.py?querystring=science

That query should return a single entry with a name containing the word science. It may be difficult to read on the command line since it is HTML. These look nicer via a GUI web browser anyway, and you can modify the filter using the form at the top of the table:


That’s it!

You now have the components of a rudimentary three-tier web application that you can use in your lab. I hope this build has provided some useful tools for you. In the final post, I will use this set of VMs and cover an example of how to implement a pool of webservers in front of the application and database tier.

Thank you for reading!

Oh, just one more thing…

Notice the pretty green lock next to the URL in my web browser in the previous screen shot?

SSL Certificate Trust

In this application, we have a self-signed SSL certificate. It should be created with the name webapp.corp.local, or whatever you selected for your environment. To get rid of the web browser security warnings and have the shiny green lock show up, you need to do two things:

Configure DNS Records

The only record you need from the client side is one that points to webapp.corp.local.

If you have a Windows-based DNS server, you can create the records using PowerShell. The following 2 lines create a DNS host (A) record for web-01a.corp.local and then a DNS alias (CNAME) record for webapp.corp.local that points to it.

PS> Add-DnsServerResourceRecordA -ComputerName 'dns.corp.local' -ZoneName 'corp.local' -name 'db-01a' -IPv4Address '' -CreatePtr

PS> Add-DnsServerResourceRecordCName -ComputerName 'dns.corp.local' -ZoneName 'corp.local' -Name 'webapp' -HostNameAlias 'web-01a.corp.local.'

This configuration allows the virtual name webapp to be separate from the web-01a name and enables the addition of other web servers to a pool, followed by the reassignment of the webapp name to a load balancer IP.

If you don’t have Windows-based DNS, you can edit your /etc/hosts file on the client or add the DNS records to your nameserver using the procedures required for your environment.

Trust the Self-Signed Certificate

Once you have name resolution knocked out, you need to trust the certificate on your client. You can really trust the certificate, or you can sometimes create an exception in the web browser. Do whichever works for you and makes you happy. Without trust, this is what the connection looks like in Chrome:


In our labs, we download the web server’s certificate to the client machine and add it to the Windows Trusted Root Certification Authorities store or one of the subtrees within Keychain Access on MacOS. That will handle IE on Windows, and Chrome browsers on Windows and MacOS.

If you save the certificate file to your desktop in Windows and double-click it, the bold text pretty much sums up what you need to do.


There are a variety of ways to get this done, and there are some shortcuts, but the process has not changed in many years and this Microsoft Windows Blog article covers a process that works.

Firefox manages its own trust store, so you need to import it separately if you want to use that browser. Check the Mozilla Wiki for detailed instructions about how to do this. Note that newer versions of Firefox have implemented more strict checking. Basically, they refuse to accept a “leaf” certificate that is specified as a Certificate Authority certificate (why is your web server using the CA certificate??) and will not allow a non-CA certificate to be added to its trusted root CA certificate store. Getting this to look nice requires additional hoops that are beyond the scope of this article. We have a Microsoft CA implemented in our labs and generally issue certificates from there. Since that CA is trusted by all clients within the environment, there is no issue.

Thank you again for reading!

You can finish the series with the last post: Use Cases.

HOL Three-Tier Application, Part 3 – App Server

This is the third post in the series about building a three-tier application for demonstration, lab, and education purposes. At this point, you have downloaded Photon OS and prepared the base template according to the specifications and have built and tested the database server, db-01a.

Working from the bottom-up, we move to the application server, which sits between the database and the web tier. As with the database, this is nothing fancy, but it gets the job done.

The Application Server (app-01a)

The application server sits in the middle of our stack and handles the formatting of data that is returned from the database. We will again use Apache and Python because we know how to use those tools from the database server build. This time, the CGI script presents an HTML form and table containing data pulled from the database via HTTP.

To add a bit of realism and fun to the process, we will configure SSL on the application server. The basic configuration is pretty simple once you have the certificate and key, and it is good to know how to do. Sure, we will be using our own self-signed certificate, but this is a lab!

The red box in the following diagram highlights the component that we are building in this post.


Note that the first steps look quite a bit like the steps we performed for the database server. I’ll outline them here. Details can be found in my previous post. Let’s get started!

  1. Deploy a copy of the base Photon template you created by following the steps in my first post. Name it something that makes sense to you for your app server. I called mine app-01a
  2. Power it up and log in as the root user
  3. Change the hostname in /etc/hostname and in /etc/hosts
  4. Change the IP address in /etc/systemd/network/10-static-eth0.network
  5. Use a SSH client to access the machine as root (this makes pasting commands easier)
  6. Install the Apache web server

Install the Apache server here, just like on the database server

# tdnf install httpd

Due to the SSL requirement on this machine, there are more files to edit than in the database server build, so get your editor ready!

Prepare to create the certificate and key

Creating the certificate is easiest if we pass it a file containing all of the answers needed by openssl. Navigate to /tmp and create the file app-01a.conf

# cd /tmp 
# vi app-01a.conf

Put the following contents into the file. Substitute values appropriate for your environment. The CN and the values under the [alt_names] section are the most important for our purposes here.

distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
C = US
ST = California
L = Palo Alto
O = VMware
OU = Hands-on Labs
CN = app-01a.corp.local
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
DNS.1 = app-01a.corp.local
DNS.2 = app-01a
IP.1 =

Save and close the file.

Generate the key and certificate from the configuration file

Note that this is a long command. You may need to scroll all the way to the right to get all of it.

# openssl req -x509 -nodes -days 1825 -newkey rsa:2048 -keyout app.key -out app.pem -config app-01a.conf

(optional) Validate that the PEM file “looks right”

# openssl x509 -in app.pem -noout -text

Read through the text output and make sure the certificate looks as you expect. If you filled out the configuration file properly, it should be fine.

Leave those files where they are for now. We’ll need them in a few minutes.

Configure the web server to run SSL on port 8443

SSL over the default port is so yesterday. Besides, we want something that differentiates this traffic from the port 443 SSL traffic that comes from the client to the web server. Changing the port makes the firewall rules more interesting and easier to distinguish. Besides, this is not normally user-facing, so there is no complexity with requiring normal users to append “:8443” to the end of the host portion of the URL.

Open the configuration file

Use the “+52” to jump to the line we want.

# vi +52 /etc/httpd/httpd.conf

Disable listening on port 80

We don’t need this server listening on the default HTTP port, so put a # in front of the “Listen 80” on line 52)

#Listen 80

Load the SSL module

Is this thing on? We need to turn on the SSL functionality, which is disabled in the default configuration. Uncomment (remove the # from the beginning of) line 143:

LoadModule ssl_module /usr/lib/httpd/modules/mod_ssl.so

Load the CGI module

Just like on the database server, we have to enable the CGI module. Add the following text on its own line at the end of the module loading, above the <IfModule unixd_module> text. I use line 176.

LoadModule cgi_module /usr/lib/httpd/modules/mod_cgi.so

Include the SSL configuration file

Near the bottom of the file (around line 508), un-comment the line

Include conf/extra/httpd-ssl.conf

Save the file and exit.

Configure SSL Settings

Open the SSL “extra settings” file.

# vi +36 /etc/httpd/conf/extra/httpd-ssl.conf

On line 36, locate the Listen line and change the port number from 443 to 8443

Listen 8443

To make things work, I also had to comment out line 92:

#SSLSessionCache "shmcb:/etc/httpd/logs/ssl_scache(512000)"

Change the virtual host’s port number from 443 to 8443 on line 121

<VirtualHost _default_:8443>

On lines 144 and 154, ensure that the following are set (these should be the defaults) and note the paths:

SSLCertificateFile "/etc/httpd/conf/server.crt"
SSLCertificateKeyFile "/etc/httpd/conf/server.key"

Put the certificates in the right place

Remember the files you created at the beginning? Now is the time to move them to where they are expected. Just overwrite anything is already there.

# mv /tmp/app.key /etc/httpd/conf/server.key
# mv /tmp/app.pem /etc/httpd/conf/server.crt

Start the web server and set it to startup when the VM boots

# systemctl start httpd
# systemctl enable httpd

Everything should start up cleanly. If not, you may have a typo somewhere. The following command provides some good information to help locate the issue:

# systemctl status -l httpd.service

Once you have the web server started properly, take a little break. Now is just the configuration part.

Create the Python script that is our “application”

As I have written previously, Python is not my native language, but it is already there and I can convince it to do what I need. This CGI script takes the text data off the HTTP connection from the database server and formats it. It also has a simple filter field and button to change the data view a bit. Each button click submits a new request to the database and shows that the connection is still possible.

I have added some complexity here because it can provide useful information in some demonstration cases. If the IP address of the source webserver is in the webservers table, its name will be displayed. Otherwise, the IP address is used when the Accessed via: line is printed. In certain situations, this can be useful for demonstrating that load balancing is working.

Create the script

# vi /etc/httpd/cgi-bin/app.py

Add the following contents to app.py, remembering that spacing is important:

#!/usr/bin/env python
import os, sys, cgi
import requests

webservers = {

print "Content-type:text/html\n\n";
print "<head><title>Customer Database</title></head>\n"
print "<body>\n"
print "<h1>Customer Database Access</h1>\n"

remote = os.getenv("REMOTE_ADDR")
form = cgi.FieldStorage()
querystring = form.getvalue("querystring")

if remote in webservers :
   accessName = webservers[remote]
else :
   accessName = remote

print "Accessed via:",accessName,"\n<p>"

if querystring != None:
   url = 'http://db-01a.corp.local/cgi-bin/data.py?querystring=' + querystring
   url = 'http://db-01a.corp.local/cgi-bin/data.py'
   querystring = ""

r = requests.get(url)

print '<form action="/cgi-bin/app.py">'
print ' Name Filter (blank for all records):'
print ' <input type="text" name="querystring" value="'+querystring+'">'
print ' <input type="submit" value="Apply">'
print '</form>'

print "\n<table border=1 bordercolor=black cellpadding=5 cellspacing=0>"

print "\n<th>Rank</th><th>Name</th><th>Universe</th><th>Revenue</th>"

#deal with the data coming across the wire
a = r.text.split("|\n#")
for row in a:
   if len(row) != 1:
      print "<tr>"
      splitrow = row.split("|")
      for item in splitrow:
         if item != None:
            print "<td>",item,"</td>"
      print "</tr>\n"
   print "</body></html>\n"

Save and close the file.

Set execute permissions

# chmod 755 /etc/httpd/cgi-bin/app.py


Now, as long as your database server is online and reachable, accessing the script via https on port 8443, will produce the data formatted as HTML:

# curl -k https://app-01a:8443/cgi-bin/app.py

If you access it via a graphical browser like Chrome or Firefox, you can and witness the fruits of your labors thus far. Note that you will have to accept the self-signed certificate, at least temporarily. The -k switch to curl tells it to ignore the untrusted SSL certificate and continue in “insecure” mode.

Here is what it looks like from Chrome. I used the IP address in the URL here because I have not created a DNS entry for app-01a. My plan is to provide access to this application only via the web front-end and I am only accessing the app server directly as part of the build.


The good news is that we are almost finished with the application. As you can see, you can use this as it stands as a “two-tier application” by creating a DNS record for the app-01a server, but I promised you three tiers. Stick with me. We cover that in the next post.

Thanks for reading!

HOL Three-Tier Application, Part 2 – DB server

This is the second post in the series about building a three-tier application for demonstration, lab, and education purposes. At this point, you have downloaded Photon OS and prepared the base template according to the specifications. If not, please go to the first post in the series to prepare your environment.

I am trying to release these posts on Wednesdays, but yesterday was a holiday in the US, so I had some time to get this one out the door early.

For the build, I will work bottom-up so it is possible to test the components as we go along and address any issues before we get to the end and have to worry about the whole stack.

The Database (db-01a)

There are many choices that could have been made here, but I wanted to minimize the steps required and use the tools I had available by default, wherever possible. So, the database server for this application uses a SQLite database. This may seem like an odd choice since it has no native remote access. But, the sqlite3 is already present on the Photon image. So is Python. Don’t expect elegance here since I do not normally use Python. I hacked together a simple CGI script to query the database and send the results over HTTP. It is not fancy, but it is small and does just enough for my needs.

Fear not! SQLite is simple enough that you don’t need any kind of DBA skills to use it. The minimal SQL that I know was enough to get this done.

The red box in the following diagram highlights the component that we are building in this post.

Application - Database

Let’s get started!

Deploy a copy of the base template

Deploy a copy of the base Photon template you created by following the steps in my first post. Name it something that makes sense to you. I called mine db-01a.


Power it up and log in as the root user.

Set the IP address

You need to configure the IP address for this machine. Here are the contents of my /etc/systemd/network/10-static-eth0.network file.



Set the name

Update the name in the /etc/hosts and /etc/hostname files with the proper name for this VM. Replace every occurrence of the name you used in the template with the name of this VM. Note that the hostname within the prompt string will change when you log out.

Restart the network to apply the settings

# systemctl restart systemd-networkd

Once you have finished with these steps, make sure you can access the network before moving on. You need Internet access in order to proceed.

SSH in

At this point, I strongly suggest you SSH to the machine so that you can paste text. Doing all of the work from the console is possible, but pasting is easier. So, fire up puTTY or whatever else you use as your preferred SSH client.

Install the web server

I am using Apache here since I can get the base install to do what I need in pretty short order. Unfortunately, it is not already present, so we need to install it. Fortunately, it only takes a few minutes:

# tdnf install httpd


Make a directory to store the database file and make apache the owner

So you don’t absolutely need to do this, but I sometimes like to separate my data from the executables.

# mkdir /etc/httpd/db
# chown apache:apache /etc/httpd/db

Start the web server and set it to startup when the VM boots

# systemctl start httpd
# systemctl enable httpd

Create the database’s front-end CGI script

There is not much to this one. It performs a simple query of the database and dumps the result. You can type in the code, but if you have connected via SSH, you can paste it in. I recommend the latter. Keep in mind that Python uses whitespace to give the program structure, so indenting is important. More precisely, the exact amount of indentation does not matter, but the relative indentation of nested blocks to one another matters a lot. If that sounds confusing, just make sure the spacing looks like mine.

This script takes an optional parameter named querystring that allows the data to be filtered on the name property of the records. It is a step above the “dump everything” approach we used in previous versions and provides the possibility for some user interaction.

Open a new file, /etc/httpd/cgi-bin/data.py

#!/usr/bin/env python
import cgi
import sqlite3


print "Content-type:text/plain\n\n";

form = cgi.FieldStorage()
querystring = form.getvalue("querystring")
if querystring != None:
   queryval = "%" + querystring + "%"
   select = "SELECT * FROM clients WHERE name LIKE '" + queryval + "'"
   select = "SELECT * FROM clients"

for row in curs.execute(select):
   if len(row) == 4:
      for item in row:
        print item,'|'
      print "#"


Save and close the file, then mark it executable

# chmod 755 /etc/httpd/cgi-bin/data.py

Create the database file and load it with data

SQLite will create the file if it is not already present. Bonus!

# sqlite3 /etc/httpd/db/clients.db

At the sqlite> prompt, create the table:

CREATE TABLE 'clients' (
 "Rank" integer,
 "Name" varchar(30),
 "Universe" varchar(25),
 "Revenue" varchar(20)

Then, load in some data. Feel free to use whatever you like:

 (1,'CHOAM','Dune','$1.7 trillion'),
 (2,'Acme Corp.','Looney Tunes','$348.7 billion'),
 (3,'Sirius Cybernetics Corp.',"Hitchhiker's Guide",'$327.2 billion'),
 (4,'Buy n Large','Wall-E','$291.8 billion'),
 (5,'Aperture Science, Inc.','Valve','$163.4 billion'),
 (6,'SPECTRE','007','$157.1 billion'),
 (7,'Very Big Corp. of America','Monty Python','$146.6 billion'),
 (8,'Frobozz Magic Co.','Zork','$112.9 billion'),
 (9,'Warbucks Industries',"Lil' Orphan Annie",'$61.5 billion'),
 (10,'Tyrell Corp.','Bladerunner','$59.4 billion'),
 (11,'Wayne Enterprises','Batman','$31.3 billion'),
 (12,'Virtucon','Austin Powers','$24.9 billion'),
 (13,'Globex','The Simpsons','$23.7 billion'),
 (14,'Umbrella Corp.','Resident Evil','$22.6 billion'),
 (15,'Wonka Industries','Charlie and the Chocolate Factory','$21.0 billion'),
 (16,'Stark Industries','Iron Man','$20.3 billion'),
 (17,'Clampett Oil','Beverly Hillbillies','$18.1 billion'),
 (18,'Oceanic Airlines','Lost','$7.8 billion'),
 (19,'Brawndo','Idiocracy','$5.8 billion'),
 (20,'Cyberdyne Systems Corp.','Terminator','$5.5 billion'),
 (21,'Paper Street Soap Company','Fight Club','$5.0 billion'),
 (22,'Gringotts','Harry Potter','$4.4 billion'),
 (23,'Oscorp','Spider-Man','$3.1 billion'),
 (24,'Nakatomi Trading Corp.','Die-Hard','$2.5 billion'),
 (25,'Los Pollos Hermanos','Breaking Bad','$1.3 billion');

Once you are happy with the data you have entered — ensure that you finish with a semi-colon and a newline — press Control-D to close the SQLite session.

Set the database file’s owner

The apache user running the web server needs access to this file in order for the CGI script to read the data.

# chown apache:apache /etc/httpd/db/clients.db

Enable CGI on the webserver

The default Apache install on Photon does not have the CGI module loaded. It is simple enough to turn it on:

Open the web server’s configuration file. The +176 before the file name opens the file at line 176, which is where we want to start:

# vi +176 /etc/httpd/httpd.conf

At line 176, add the following line to load the CGI module:

LoadModule cgi_module /usr/lib/httpd/modules/mod_cgi.so

At line 379, add the following to enable access to the database directory. It goes right before a line that starts with <IfModule mime_module>

#database directory
<Directory "/etc/httpd/db">
    AllowOverride None
    Options None
    Require all granted

Save and close the file.

Restart the web server to read the updated configuration

# systemctl restart httpd


Now, you can access the script via http and you should see the data.

# curl http://db-01a/cgi-bin/data.py

It won’t look too pretty, but the user never sees this back end data. That’s where the application server comes in. At this point, the result should look something like this:

root@db-01a [ ~ ]# curl http://db-01a/cgi-bin/data.py
1 |
Dune |
$1.7 trillion |
... (truncated) ...
24 |
Nakatomi Trading Corp. |
Die-Hard |
$2.5 billion |
25 |
Los Pollos Hermanos |
Breaking Bad |
$1.3 billion |
root@db-01a [ ~ ]#

The next piece of the puzzle is the application server, which consumes the data provided by this component. If you had no problems with this setup, the rest should be a breeze. This may be the most complicated part of the whole application.

Thank you for reading!

The next post detailing the build of the App server is available here.

HOL Three-Tier Application, Part 1

This is the first post in my series about the multi-tier application we use in some of the VMware Hands-on Labs to demonstrate, among other things, network connectivity, microsegmentation and load balancing. This post will cover downloading the base operating system and performing the configuration tasks common to all of the VMs in the solution. As with anything, there are multiple ways to do this. This represents the way that worked for me.

The Need

Whether you live in a cutting-edge, microservices-oriented world, or have a traditional application spread across multiple machines, the components (machines, containers, services, processes, etc.) need to communicate with one another over the network. Understanding what that looks like is important to securing the connection end-to-end. This simple application is intended to provide a starting point for learning or testing firewall and load balancing configurations to see how they affect a distributed environment.

For instruction purposes, we wanted three simple, independent parts that could be deployed, rearranged, and otherwise manipulated to illustrate many different situations that may occur in an environment. For HOL and other labs, small is usually good. Oh, and fast. It should be fast.

The Application

This application consists of three operating system instances, independent VMs, each of which handles a specific task. When all of them can communicate over the network over the required ports, the client receives the requested information and can interact with that information. If there is a breakdown, not so much.

This demonstration application has been created so that each component VM is independent from the others: IP addresses can be changed and multiple instances of the web and application tier VMs can be created by cloning, renaming, and re-addressing. The basic build with one of each type and all resources on the same subnet will be described in this series. The following is a simple diagram of what I will be covering. I put SSL in here because it is always a good idea to secure your web traffic, and it provides the opportunity to configure a load balancer in front of the web tier in a more realistic scenario.


So, let’s get to it!

Build The Base

This application is built using VMware’s Photon OS. If you are not familiar with the Photon project, you can read more on the VMware Photon OS page. Basically, as the page indicates, Photon OS is a Minimal Linux Container Host. Because we have very basic needs, we are going to focus on the first half (Minimal Linux) and ignore the second half (Container Host) for now. One really cool thing about Photon OS is that it boots incredibly quickly.

Before we do anything, I’d like to give you an idea of the time involved in building this application. Once I have the software downloaded and have staged the base Photon template, I can get the basic application up, running, and captured in under an hour. If you are comfortable using the vi editor and an SSH connection, I think you can as well. Even if you are a bit rusty, it should not take too much longer than that. My time is skewed a bit since I was documenting the build.

Download the Software

This application is going to run as a set of virtual machines on my VMware ESXi hosts. I selected the Photon OS, Version 1.0 — OVA with virtual hardware v10 as my starting point. If you like, you can install Photon on your own from the ISO, but this has nearly everything we need in a simple package: A pre-installed and vSphere-optimized Photon OS minimal instance configured with virtual hardware version 10. At the time of this writing, that file was available using the link at the bottom of the VMware Photon OS page. The file I downloaded was called photon-custom-hw10-1.0-13c08b6.ova and is less than 300 MB.

Import the OVA

Once you have downloaded the software, import the OVA to your environment and power it up.

Create a Baseline

You can handle this however you like, but I have some tasks that are common across all of the VMs and don’t like to duplicate work if I can avoid it. Note that you will need Internet access from the VM in order to install software. You will also need three IP addresses that you can statically assign to the VMs.

Set the root password

The default password on the OVA is changeme — use this to log in with the user name root. The system will prompt you again for the changeme password and then require you to set a complex password. It didn’t like our standard (simple) HOL preferred password, so I had to set it to VMware123! and then I used passwd to change it to VMware1! that we use in all of the Hands-on Labs. Note that passwd will complain about a weak password, but will still let you change it here as long as you are persistent:


Ensure that root’s password does not expire

It is always a drag when you finally get back to working on your lab, only to have to reset passwords… or, worse, figure out how to break in because the password is no longer valid. In production, I probably would not do this, but this is a lab tool.

Note that my convention is to prefix the examples with a “#” because they are executed as the root user. You don’t type the “#”

# chage -M -1 root

Note that is a NEGATIVE ONE after the -M

Set the hostname

Change the hostname from the default generated name to what you want to use. For the template, I usually set it to something besides the default  photon-<some random characters> so that I know I have done this work. Note, if you’re not familiar with the vi editor, look here for a “cheat sheet” from MIT.

# vi /etc/hostname

Replace the current name with the new name and save, close the file.

Set a static IP (change from default DHCP)

In this OVA, the default network configuration is stored in /etc/systemd/network/10-dhcp-en.network. To configure a static IP address on the eth0 interface, rename the file and replace the contents:

# mv /etc/systemd/network/10-dhcp-en.network /etc/systemd/network/10-static-eth0.network

Renaming it instead of copying it retains the permissions so that it will work. The contents are pretty straightforward. The following example is for the web-01a machine in my environment. Substitute with addresses that make sense for you. Don’t count on DNS to work once these VMs are deployed in DMZs or microsegments, but I configure it because I need to be able to resolve repository hostnames to install software:



Restart the network to apply the settings

# systemctl restart systemd-networkd

Edit the hosts file

Because this application is intended to be self-contained, we use local hosts files for name resolution. Configuring this template with all of the names and IPs that you want to use is easier than doing it later for each VM. Specifying names allows the other tools’ configurations to be built using names instead of IP addresses. This and makes changing addresses later much easier.

Remember to also change the hostname on the loopback ( from the default to your host’s name, too. This is an example of the edited file from our web-01a machine:

# Begin /etc/hosts (network card version)

::1 localhost ipv6-localhost ipv6-loopback localhost.localdomain localhost web-01a
# End /etc/hosts (network card version) db-01a.corp.local  db-01a app-01a.corp.local app-01a web-01a.corp.local web-01a

Modify the firewall to allow the desired ports

The iptables config script run at startup of the Photon OS is /etc/systemd/scripts/iptables and only allows SSH by default. Add the following lines to the bottom of the file:

#Enable ping by anyone
iptables -A INPUT -p icmp -j ACCEPT

#Enable web and app traffic 
iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT

The last three open the ports we need for all of the app layers. You can comment out the ones you don’t need for each VM after you deploy each one… or not.

Restart iptables to apply the new rules

# systemctl restart iptables

(optional) Verify the new rules

# iptables -L

(optional) Enable key-based SSH

If you have an SSH key that you use, now is a good time to copy your SSH key to the /root/.ssh/authorized_keys file, replacing the <ssh-key-here> text that is there by default.

(optional) Install software used by all

The OVA contains a minimal installation of Photon OS, but I created this application with the default packages in mind. We use the tdnf tool to perform installations on Photon. While adding lsof is optional, I find it excellent for troubleshooting.

# tdnf install lsof

Once installed, try this to see which services are listening and connected on which ports:

# lsof -i -P -n

Cool, right?

If you have anything else that you want to install — say you prefer nano to vim as a text editor — go ahead and install that now using the same tdnf syntax:

# tdnf install nano

Finish Up

I usually reboot here just to make sure everything comes up as expected before moving on. With Photon, that reboot only takes a few seconds.

If everything looks good, shut this machine down and clone it to a template for use when creating the web, app, and database server machines. For this example, I called mine photon:


Next time, I will cover the build of the database VM using this template as a starting point.

Thank you for reading!