By Gina Daly, Technical Writer for Technical Marketing, End-User Computing, VMware
With significant contributions from:
Stephane Asselin, Senior End-User-Computing Architect, Technical Enablement, End-User Computing, VMware
Jason Marshall, Manager, Product Engineering, End-User Computing, VMware
Andrew Johnson, Senior Manager and Staff Architect, End-User-Computing Technical Marketing and Enablement, VMware
My father was a naval officer and taught me many things about the sea. Did you know, in comparison to the size of a ship, only a very small helm or wheel is needed to steer this ship? In an App Volumes environment with various components, it can sometimes be difficult to know where to start troubleshooting. We have recently published a number of Knowledge Base (KB) articles to steer you in the right direction, just like that helm. Our aim is to simplify App Volumes troubleshooting and help pinpoint the issue, should you encounter stormy waters!
General App Volumes Troubleshooting Overview
The following diagram provides a list of general troubleshooting steps. In this blog, we discuss the first four steps. Within each step description, links to the corresponding KB articles are included. Using the Ruby on Rails console and SQL database will be covered in the next blog post, How to Troubleshoot an App Volumes Environment, Part 2.
App Volumes Troubleshooting Steps
- Check deployment requirements and port connectivity.
It is vital to know the system requirements for installation and deployment. A solid structure needs a solid foundation, and the deployment requirements are the foundation for your App Volumes environment. The KB article VMware App Volumes deployment requirements has a useful reference table listing the basic deployment requirements, along with other items to check, such as Active Directory (AD) and vCenter Server permissions.
Port connectivity is equally important, particularly if a firewall exists between system components. Network connectivity requirements for App Volumes 2.5.x and later lists the necessary ports. You can eliminate simple misconfiguration issues by checking the basics.
Another handy KB for checking App Volumes configurations is Confirming VMware App Volumes environment information from registry entries. You can easily scan details such as the product version, SQL database name, and HTTP ports used, all from a single location in the registry.
- Check the App Volumes Manager console—specifically the Dashboard and Activity tabs.
There are two tabs on the App Volumes Manager console that give you quick insight into possible issues in your environment. At a glance, the Dashboard tab displays information such as how many licenses are in use, AppStack utilization details, and most recent events in your environment. Hovering the mouse over any of these tables displays real-time statistics. From the Activity tab, you can access the activity and server log files without having to generate a log bundle. System errors can be seen from the System Messages tab within the Activity tab.
VMware App Volumes Activity Tab
More details are in the KB article Troubleshooting VMware App Volumes through the App Volumes Manager console.
- Use a log analysis tool to check various log files.
Scrolling through log files can be a tedious task. Using a log analysis tool such as BareTail, or even a text editor such as Notepad++, can simplify this task.
Analyzing Log File with BareTail
Use these tools to investigate the App Volumes Manager production log, the App Volumes Agent logs, and the ESXi service log. The KB article Location of VMware App Volumes log files describes how and where to find the relevant logs, and VMware log analysis tools and log analysis examples gives examples of how to use the log analysis tools to solve problems.
- Understand what should be in the logs.
Of course, when searching through log files, it is helpful to know what you are looking for, what is expected behavior, and where the red flags are. We have all been in this predicament—we diligently search the log files for an “error” string, find the “error,” and are still none the wiser after the fact! VMware log analysis tools and log analysis examples describes the relevant search terms, how to recognize an error, and even provides log analysis examples for you to follow.
Let us take a look at the production.log file on the App Volumes Manager server, which contains logging from the main App Volumes modules. Here are the App Volumes modules you can search for, along with their descriptions.
- RADIR: RubyActiveDIRectory classes – Responsible for AD connection
- Cvo: App Volumes logic classes
- RvSphere: RubyvSphere wrapper classes
- SQL: Database queries (Default logging only captures errors. Debug logging captures queries and errors.)
- PowerShell: App Volumes PowerShell interface classes
- NTLM: App Volumes NTLM handler
In the following log excerpt, we can see the RADIR module next to a successful AD connection message.
Analyzing App Volumes Production Log
Each line in the log file gives us the process ID (PID) on the left side, and tells us whether it is an active request (PxxxxRxx) or a delayed job (PxxxxDJ).
Checking Process ID in App Volumes Production Log
To whet your appetite, here is a list of the log analysis examples in VMware App Volumes log analysis tools and log analysis examples. In each situation, we look at what is recorded in the logs, so you can determine the point of failure when provisioning an AppStack, for example. In the case of an unreachable component, we discuss the observed behavior in your environment, and which log file and search string to check for in each scenario.
- Provision a new AppStack and monitor progress
- Create a new writable volume and monitor progress
- App Volumes Manager unreachable
- SQL database unreachable
- Active Directory unreachable
- vCenter Server unreachable
In summary, here are the steps and relevant KB articles to follow when troubleshooting your App Volumes issues.
- Check deployment requirements and port connectivity.
- Check the App Volumes Manager console — specifically the Dashboard and Activity tabs.
- Use a log analysis tool to check various log files.
- Understand what should be in the logs.
Sail on to part 2 of this blog, How to Troubleshoot an App Volumes Environment, Part 2, where we continue our journey and cover the remaining steps: using the Ruby on Rails console and SQL database to troubleshoot an App Volumes environment.
