This is the fourth post in the series about building a three-tier application for demonstration, lab, and education purposes. If you have been following along, you have created the base Photon template as well as simple database and application servers.

This post covers the final layer of our stack, the web presentation tier. I have said it before, but the configuration of the web server here is really simple, and if you have made it this far, you’re golden.

The Web Server (web-01a)

All of the hosts in this application run “web server” software, but this one has the web server designation because it is the one that the user directly accesses. The entire back end could be replaced with real application middleware and an RDBMs, but the user expects this one to present an SSL-encrypted web page on port 443. This time, I have chosen not to use Apache, and there is no need for Python. There is no CGI, and minimal configuration is required aside from issuing another certificate for the SSL. This tier is mostly interesting because it will support the virtual name of the application in addition to the real name(s) of your web server(s).

The red box in the following diagram highlights the component that we are building in this post.

Again, the first steps look quite a bit like the steps we performed for the application and database servers as we assign the personality to the template. I will again outline the steps here as a reminder. Details can be found in my post about the database server.

Let’s get started!

1. Deploy a copy of the base Photon template you created by following the steps in my first post.
2. Name it something that makes sense to you for your web server. I called mine web-01a
3. Power it up and log in as the root user
4. Change the hostname in /etc/hostname and in /etc/hosts
5. Change the IP address in /etc/systemd/network/10-static-eth0.network
6. Use a SSH client to access the machine as root (makes pasting possible)

Instead of installing Apache again, we are going to use nginx. You can do the same thing with Apache, but I wanted to try something a little more lightweight and the nginx configuration for this use case is really simple.

Install nginx to be used as a reverse proxy

The web server machine will function as a reverse proxy, sending user requests bound for port 443 on this server to the application server at https://app-01a.corp.local:8443

# tdnf install nginx

The nginx install is less than 6 MB and takes a few seconds:

Configure the reverse proxy

Edit the configuration file, /etc/nginx/nginx.conf

# vi +116 /etc/nginx/nginx.conf

by adding the following at the bottom of the file, at line 116, just before the closing “}” in the file.

   # HTTPS server
   #
   server {
      listen 443;
      server_name webapp.corp.local;

      ssl on;
      ssl_certificate     /etc/nginx/ssl/webapp.pem;
      ssl_certificate_key /etc/nginx/ssl/webapp.key;

      ssl_session_cache shared:SSL:1m;
      ssl_session_timeout 2m;

      location / {
         proxy_pass https://app-01a.corp.local:8443/;
         proxy_set_header Host $host;
         proxy_redirect http:// https://;
      }
   }

Notice that we need an SSL certificate and a key to make this work. We have done this before, so let’s create those now.

Make the ssl directory and switch into it

Let’s just create the certificates in the place that the server expects them to be.

# mkdir -p /etc/nginx/ssl
# cd /etc/nginx/ssl

Build the configuration file

It is common for multiple web servers to be configured in a pool, behind a load balancer. I create the certificate here using a name, webapp.corp.local. This name can be assigned to the load balancer’s VIP. If there is only one web server, as in my example here, this name can also be an alias that resolves to the one web server. For simplicity, and possibly for other use cases, the certificate configuration we build here includes the names of three web servers: web-01a, web-02a and web-03a.

Create the file webapp.conf

# vi webapp.conf

with the following contents, modified as needed for your environment:

[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
default_md = sha256
prompt = no
[req_distinguished_name]
C = US
ST = California
L = Palo Alto
O = VMware
OU = Hands-on Labs
CN = webapp.corp.local
[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = webapp.corp.local
DNS.2 = web-01a.corp.local
DNS.3 = web-02a.corp.local
DNS.4 = web-03a.corp.local

Save and close the file.

Generate the key and certificate

Note that this is a long command and you may need to scroll to the right to get all of it. Ensure it ends with “webapp.conf”

# openssl req -x509 -nodes -days 1825 -newkey rsa:2048 -keyout webapp.key -out webapp.pem -config webapp.conf

(optional) Validate that the PEM file “looks right”

I put this command here for those who want to look at the certificate. It is a good command to know in case you have a certificate file and want to know what information it contains. That can help you match the certificate to the proper host without needing to install it and then find out it is not the right one.

# openssl x509 -in webapp.pem -noout -text

Start the nginx server and configure it to startup when the VM boots

# systemctl start nginx
# systemctl enable nginx

Verify

With the other components (db-01a, app-01a) online, reachable and tested, you can test the whole solution with curl from the console of the web server

# curl -k https://web-01a/cgi-bin/app.py

This should return the data from the database in HTML format by executing the script on the application server.

You can filter the results by appending a querystring. Try this one:

# curl -k https://web-01a/cgi-bin/app.py?querystring=science

That query should return a single entry with a name containing the word science. It may be difficult to read on the command line since it is HTML. These look nicer via a GUI web browser anyway, and you can modify the filter using the form at the top of the table:

That’s it!

You now have the components of a rudimentary three-tier web application that you can use in your lab. I hope this build has provided some useful tools for you. In the final post, I will use this set of VMs and cover an example of how to implement a pool of webservers in front of the application and database tier.

Thank you for reading!

Oh, just one more thing…

Notice the pretty green lock next to the URL in my web browser in the previous screen shot?

SSL Certificate Trust

In this application, we have a self-signed SSL certificate. It should be created with the name webapp.corp.local, or whatever you selected for your environment. To get rid of the web browser security warnings and have the shiny green lock show up, you need to do two things:

Configure DNS Records

The only record you need from the client side is one that points to webapp.corp.local.

If you have a Windows-based DNS server, you can create the records using PowerShell. The following 2 lines create a DNS host (A) record for web-01a.corp.local and then a DNS alias (CNAME) record for webapp.corp.local that points to it.

PS> Add-DnsServerResourceRecordA -ComputerName 'dns.corp.local' -ZoneName 'corp.local' -name 'db-01a' -IPv4Address '192.168.120.10' -CreatePtr

PS> Add-DnsServerResourceRecordCName -ComputerName 'dns.corp.local' -ZoneName 'corp.local' -Name 'webapp' -HostNameAlias 'web-01a.corp.local.'

This configuration allows the virtual name webapp to be separate from the web-01a name and enables the addition of other web servers to a pool, followed by the reassignment of the webapp name to a load balancer IP.

If you don’t have Windows-based DNS, you can edit your /etc/hosts file on the client or add the DNS records to your nameserver using the procedures required for your environment.

Trust the Self-Signed Certificate

Once you have name resolution knocked out, you need to trust the certificate on your client. You can really trust the certificate, or you can sometimes create an exception in the web browser. Do whichever works for you and makes you happy. Without trust, this is what the connection looks like in Chrome:

In our labs, we download the web server’s certificate to the client machine and add it to the Windows Trusted Root Certification Authorities store or one of the subtrees within Keychain Access on MacOS. That will handle IE on Windows, and Chrome browsers on Windows and MacOS.

If you save the certificate file to your desktop in Windows and double-click it, the bold text pretty much sums up what you need to do.

There are a variety of ways to get this done, and there are some shortcuts, but the process has not changed in many years and this Microsoft Windows Blog article covers a process that works.

Firefox manages its own trust store, so you need to import it separately if you want to use that browser. Check the Mozilla Wiki for detailed instructions about how to do this. Note that newer versions of Firefox have implemented more strict checking. Basically, they refuse to accept a “leaf” certificate that is specified as a Certificate Authority certificate (why is your web server using the CA certificate??) and will not allow a non-CA certificate to be added to its trusted root CA certificate store. Getting this to look nice requires additional hoops that are beyond the scope of this article. We have a Microsoft CA implemented in our labs and generally issue certificates from there. Since that CA is trusted by all clients within the environment, there is no issue.

Thank you again for reading!

You can finish the series with the last post: Use Cases.

This is the second post in the series about building a three-tier application for demonstration, lab, and education purposes. At this point, you have downloaded Photon OS and prepared the base template according to the specifications. If not, please go to the first post in the series to prepare your environment.

I am trying to release these posts on Wednesdays, but yesterday was a holiday in the US, so I had some time to get this one out the door early.

For the build, I will work bottom-up so it is possible to test the components as we go along and address any issues before we get to the end and have to worry about the whole stack.

The Database (db-01a)

There are many choices that could have been made here, but I wanted to minimize the steps required and use the tools I had available by default, wherever possible. So, the database server for this application uses a SQLite database. This may seem like an odd choice since it has no native remote access. But, the sqlite3 is already present on the Photon image. So is Python. Don’t expect elegance here since I do not normally use Python. I hacked together a simple CGI script to query the database and send the results over HTTP. It is not fancy, but it is small and does just enough for my needs.

Fear not! SQLite is simple enough that you don’t need any kind of DBA skills to use it. The minimal SQL that I know was enough to get this done.

The red box in the following diagram highlights the component that we are building in this post.

Let’s get started!

Deploy a copy of the base template

Deploy a copy of the base Photon template you created by following the steps in my first post. Name it something that makes sense to you. I called mine db-01a.

Power it up and log in as the root user.

Set the IP address

You need to configure the IP address for this machine. Here are the contents of my /etc/systemd/network/10-static-eth0.network file.

[Match]
Name=eth0

[Network]
Address=192.168.120.10/24
Gateway=192.168.120.1
DNS=192.168.110.10
Domains=corp.local

Set the name

Update the name in the /etc/hosts and /etc/hostname files with the proper name for this VM. Replace every occurrence of the name you used in the template with the name of this VM. Note that the hostname within the prompt string will change when you log out.

Restart the network to apply the settings

# systemctl restart systemd-networkd

Once you have finished with these steps, make sure you can access the network before moving on. You need Internet access in order to proceed.

SSH in

At this point, I strongly suggest you SSH to the machine so that you can paste text. Doing all of the work from the console is possible, but pasting is easier. So, fire up puTTY or whatever else you use as your preferred SSH client.

Install the web server

I am using Apache here since I can get the base install to do what I need in pretty short order. Unfortunately, it is not already present, so we need to install it. Fortunately, it only takes a few minutes:

# tdnf install httpd

Make a directory to store the database file and make apache the owner

So you don’t absolutely need to do this, but I sometimes like to separate my data from the executables.

# mkdir /etc/httpd/db
# chown apache:apache /etc/httpd/db

Start the web server and set it to startup when the VM boots

# systemctl start httpd
# systemctl enable httpd

Create the database’s front-end CGI script

There is not much to this one. It performs a simple query of the database and dumps the result. You can type in the code, but if you have connected via SSH, you can paste it in. I recommend the latter. Keep in mind that Python uses whitespace to give the program structure, so indenting is important. More precisely, the exact amount of indentation does not matter, but the relative indentation of nested blocks to one another matters a lot. If that sounds confusing, just make sure the spacing looks like mine.

This script takes an optional parameter named querystring that allows the data to be filtered on the name property of the records. It is a step above the “dump everything” approach we used in previous versions and provides the possibility for some user interaction.

Open a new file, /etc/httpd/cgi-bin/data.py

#!/usr/bin/env python
import cgi
import sqlite3

conn=sqlite3.connect('/etc/httpd/db/clients.db')
curs=conn.cursor()

print "Content-type:text/plain\n\n";

form = cgi.FieldStorage()
querystring = form.getvalue("querystring")
if querystring != None:
   queryval = "%" + querystring + "%"
   select = "SELECT * FROM clients WHERE name LIKE '" + queryval + "'"
else:
   select = "SELECT * FROM clients"

for row in curs.execute(select):
   if len(row) == 4:
      for item in row:
        print item,'|'
      print "#"

conn.close()

Save and close the file, then mark it executable

# chmod 755 /etc/httpd/cgi-bin/data.py

Create the database file and load it with data

SQLite will create the file if it is not already present. Bonus!

# sqlite3 /etc/httpd/db/clients.db

At the sqlite> prompt, create the table:

CREATE TABLE 'clients' (
 "Rank" integer,
 "Name" varchar(30),
 "Universe" varchar(25),
 "Revenue" varchar(20)
 );

Then, load in some data. Feel free to use whatever you like:

INSERT INTO 'clients' VALUES
 (1,'CHOAM','Dune','$1.7 trillion'),
 (2,'Acme Corp.','Looney Tunes','$348.7 billion'),
 (3,'Sirius Cybernetics Corp.',"Hitchhiker's Guide",'$327.2 billion'),
 (4,'Buy n Large','Wall-E','$291.8 billion'),
 (5,'Aperture Science, Inc.','Valve','$163.4 billion'),
 (6,'SPECTRE','007','$157.1 billion'),
 (7,'Very Big Corp. of America','Monty Python','$146.6 billion'),
 (8,'Frobozz Magic Co.','Zork','$112.9 billion'),
 (9,'Warbucks Industries',"Lil' Orphan Annie",'$61.5 billion'),
 (10,'Tyrell Corp.','Bladerunner','$59.4 billion'),
 (11,'Wayne Enterprises','Batman','$31.3 billion'),
 (12,'Virtucon','Austin Powers','$24.9 billion'),
 (13,'Globex','The Simpsons','$23.7 billion'),
 (14,'Umbrella Corp.','Resident Evil','$22.6 billion'),
 (15,'Wonka Industries','Charlie and the Chocolate Factory','$21.0 billion'),
 (16,'Stark Industries','Iron Man','$20.3 billion'),
 (17,'Clampett Oil','Beverly Hillbillies','$18.1 billion'),
 (18,'Oceanic Airlines','Lost','$7.8 billion'),
 (19,'Brawndo','Idiocracy','$5.8 billion'),
 (20,'Cyberdyne Systems Corp.','Terminator','$5.5 billion'),
 (21,'Paper Street Soap Company','Fight Club','$5.0 billion'),
 (22,'Gringotts','Harry Potter','$4.4 billion'),
 (23,'Oscorp','Spider-Man','$3.1 billion'),
 (24,'Nakatomi Trading Corp.','Die-Hard','$2.5 billion'),
 (25,'Los Pollos Hermanos','Breaking Bad','$1.3 billion');

Once you are happy with the data you have entered — ensure that you finish with a semi-colon and a newline — press Control-D to close the SQLite session.

Set the database file’s owner

The apache user running the web server needs access to this file in order for the CGI script to read the data.

# chown apache:apache /etc/httpd/db/clients.db

Enable CGI on the webserver

The default Apache install on Photon does not have the CGI module loaded. It is simple enough to turn it on:

Open the web server’s configuration file. The +176 before the file name opens the file at line 176, which is where we want to start:

# vi +176 /etc/httpd/httpd.conf

At line 176, add the following line to load the CGI module:

LoadModule cgi_module /usr/lib/httpd/modules/mod_cgi.so

At line 379, add the following to enable access to the database directory. It goes right before a line that starts with <IfModule mime_module>

#database directory
<Directory "/etc/httpd/db">
    AllowOverride None
    Options None
    Require all granted
</Directory>

Save and close the file.

Restart the web server to read the updated configuration

# systemctl restart httpd

Verify

Now, you can access the script via http and you should see the data.

# curl http://db-01a/cgi-bin/data.py

It won’t look too pretty, but the user never sees this back end data. That’s where the application server comes in. At this point, the result should look something like this:

root@db-01a [ ~ ]# curl http://db-01a/cgi-bin/data.py
1 |
CHOAM |
Dune |
$1.7 trillion |
#
... (truncated) ...
#
24 |
Nakatomi Trading Corp. |
Die-Hard |
$2.5 billion |
#
25 |
Los Pollos Hermanos |
Breaking Bad |
$1.3 billion |
#
root@db-01a [ ~ ]#

The next piece of the puzzle is the application server, which consumes the data provided by this component. If you had no problems with this setup, the rest should be a breeze. This may be the most complicated part of the whole application.

Thank you for reading!

The next post detailing the build of the App server is available here.

This is the first post in my series about the multi-tier application we use in some of the VMware Hands-on Labs to demonstrate, among other things, network connectivity, microsegmentation and load balancing. This post will cover downloading the base operating system and performing the configuration tasks common to all of the VMs in the solution. As with anything, there are multiple ways to do this. This represents the way that worked for me.

The Need

Whether you live in a cutting-edge, microservices-oriented world, or have a traditional application spread across multiple machines, the components (machines, containers, services, processes, etc.) need to communicate with one another over the network. Understanding what that looks like is important to securing the connection end-to-end. This simple application is intended to provide a starting point for learning or testing firewall and load balancing configurations to see how they affect a distributed environment.

For instruction purposes, we wanted three simple, independent parts that could be deployed, rearranged, and otherwise manipulated to illustrate many different situations that may occur in an environment. For HOL and other labs, small is usually good. Oh, and fast. It should be fast.

The Application

This application consists of three operating system instances, independent VMs, each of which handles a specific task. When all of them can communicate over the network over the required ports, the client receives the requested information and can interact with that information. If there is a breakdown, not so much.

This demonstration application has been created so that each component VM is independent from the others: IP addresses can be changed and multiple instances of the web and application tier VMs can be created by cloning, renaming, and re-addressing. The basic build with one of each type and all resources on the same subnet will be described in this series. The following is a simple diagram of what I will be covering. I put SSL in here because it is always a good idea to secure your web traffic, and it provides the opportunity to configure a load balancer in front of the web tier in a more realistic scenario.

So, let’s get to it!

Build The Base

This application is built using VMware’s Photon OS. If you are not familiar with the Photon project, you can read more on the VMware Photon OS page. Basically, as the page indicates, Photon OS is a Minimal Linux Container Host. Because we have very basic needs, we are going to focus on the first half (Minimal Linux) and ignore the second half (Container Host) for now. One really cool thing about Photon OS is that it boots incredibly quickly.

Before we do anything, I’d like to give you an idea of the time involved in building this application. Once I have the software downloaded and have staged the base Photon template, I can get the basic application up, running, and captured in under an hour. If you are comfortable using the vi editor and an SSH connection, I think you can as well. Even if you are a bit rusty, it should not take too much longer than that. My time is skewed a bit since I was documenting the build.

Download the Software

This application is going to run as a set of virtual machines on my VMware ESXi hosts. I selected the Photon OS, Version 1.0 — OVA with virtual hardware v10 as my starting point. If you like, you can install Photon on your own from the ISO, but this has nearly everything we need in a simple package: A pre-installed and vSphere-optimized Photon OS minimal instance configured with virtual hardware version 10. At the time of this writing, that file was available using the link at the bottom of the VMware Photon OS page. The file I downloaded was called photon-custom-hw10-1.0-13c08b6.ova and is less than 300 MB.

Import the OVA

Once you have downloaded the software, import the OVA to your environment and power it up.

Create a Baseline

You can handle this however you like, but I have some tasks that are common across all of the VMs and don’t like to duplicate work if I can avoid it. Note that you will need Internet access from the VM in order to install software. You will also need three IP addresses that you can statically assign to the VMs.

Set the root password

The default password on the OVA is changeme — use this to log in with the user name root. The system will prompt you again for the changeme password and then require you to set a complex password. It didn’t like our standard (simple) HOL preferred password, so I had to set it to VMware123! and then I used passwd to change it to VMware1! that we use in all of the Hands-on Labs. Note that passwd will complain about a weak password, but will still let you change it here as long as you are persistent:

Ensure that root’s password does not expire

It is always a drag when you finally get back to working on your lab, only to have to reset passwords… or, worse, figure out how to break in because the password is no longer valid. In production, I probably would not do this, but this is a lab tool.

Note that my convention is to prefix the examples with a “#” because they are executed as the root user. You don’t type the “#”

# chage -M -1 root

Note that is a NEGATIVE ONE after the -M

Set the hostname

Change the hostname from the default generated name to what you want to use. For the template, I usually set it to something besides the default  photon-<some random characters> so that I know I have done this work. Note, if you’re not familiar with the vi editor, look here for a “cheat sheet” from MIT.

# vi /etc/hostname

Replace the current name with the new name and save, close the file.

Set a static IP (change from default DHCP)

In this OVA, the default network configuration is stored in /etc/systemd/network/10-dhcp-en.network. To configure a static IP address on the eth0 interface, rename the file and replace the contents:

# mv /etc/systemd/network/10-dhcp-en.network /etc/systemd/network/10-static-eth0.network

Renaming it instead of copying it retains the permissions so that it will work. The contents are pretty straightforward. The following example is for the web-01a machine in my environment. Substitute with addresses that make sense for you. Don’t count on DNS to work once these VMs are deployed in DMZs or microsegments, but I configure it because I need to be able to resolve repository hostnames to install software:

[Match]
Name=eth0

[Network]
Address=192.168.120.30/24
Gateway=192.168.120.1
DNS=192.168.110.10
Domains=corp.local

Restart the network to apply the settings

# systemctl restart systemd-networkd

Edit the hosts file

Because this application is intended to be self-contained, we use local hosts files for name resolution. Configuring this template with all of the names and IPs that you want to use is easier than doing it later for each VM. Specifying names allows the other tools’ configurations to be built using names instead of IP addresses. This and makes changing addresses later much easier.

Remember to also change the hostname on the loopback (127.0.0.1) from the default to your host’s name, too. This is an example of the edited file from our web-01a machine:

# Begin /etc/hosts (network card version)

::1 localhost ipv6-localhost ipv6-loopback
127.0.0.1 localhost.localdomain
127.0.0.1 localhost
127.0.0.1 web-01a
# End /etc/hosts (network card version)
192.168.120.10 db-01a.corp.local  db-01a
192.168.120.20 app-01a.corp.local app-01a
192.168.120.30 web-01a.corp.local web-01a

Modify the firewall to allow the desired ports

The iptables config script run at startup of the Photon OS is /etc/systemd/scripts/iptables and only allows SSH by default. Add the following lines to the bottom of the file:

#Enable ping by anyone
iptables -A INPUT -p icmp -j ACCEPT

#Enable web and app traffic 
iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
iptables -A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT

The last three open the ports we need for all of the app layers. You can comment out the ones you don’t need for each VM after you deploy each one… or not.

Restart iptables to apply the new rules

# systemctl restart iptables

(optional) Verify the new rules

# iptables -L

(optional) Enable key-based SSH

If you have an SSH key that you use, now is a good time to copy your SSH key to the /root/.ssh/authorized_keys file, replacing the <ssh-key-here> text that is there by default.

(optional) Install software used by all

The OVA contains a minimal installation of Photon OS, but I created this application with the default packages in mind. We use the tdnf tool to perform installations on Photon. While adding lsof is optional, I find it excellent for troubleshooting.

# tdnf install lsof

Once installed, try this to see which services are listening and connected on which ports:

# lsof -i -P -n

Cool, right?

If you have anything else that you want to install — say you prefer nano to vim as a text editor — go ahead and install that now using the same tdnf syntax:

# tdnf install nano

Finish Up

I usually reboot here just to make sure everything comes up as expected before moving on. With Photon, that reboot only takes a few seconds.

If everything looks good, shut this machine down and clone it to a template for use when creating the web, app, and database server machines. For this example, I called mine photon:

Next time, I will cover the build of the database VM using this template as a starting point.

Thank you for reading!