Analyzing Virtual Desktop Login Time

By Gourav Bhardwaj with Matt Larson

Often when performing health checks a discussion arises about the login time and what constitutes login time. This article covers some of the common ways to look at login time and its underlying components.  You can look at login time using vCOps for View or a third-party user experience monitoring solution. In this example the login time is demonstrated using Stratusphere™ UX. Experienced system administrators can also use this process to troubleshoot slow login times.

Review Virtual Desktop login times using Stratusphere UX™

1. First, ensure you are in the Stratusphere UX Interface.
   
2. On the Inspector tab, choose Machine Diagnostic Summary, and then click Go.
   
3. In the Date Range drop-down menu, select Last 24 Hours.
4. In the results list, sort by Login Delay.
5. Click the down-arrow next to the name of the machine. Click Drill-down to see machine inspection history.
   
6. Select the down-arrow next to the hour that contains the slow login time. Click Drill-down to see inspection report details.
   

A lot of information will be provided, including the username of the user experiencing the issues, as well as information regarding processes. One important piece of information used to find what may be causing the slow logins is the CPU System Time(s) field. The graphic below shows VMWVvpsvc running long. This metric indicates some login slowness resulting from the profile being copied from the profile location using VMware’s persona management. This may be the result of a file server being in a location local to the user, but not local to the View environment.

This information is helpful, as it says that the VMWVvpsvc was running for 94 seconds. We can assume this is mostly during login, but that only accounts for 94 seconds of a 351 second login delay. Clearly, more information is necessary. While turning to logs can be helpful (such as persona management, the system event log, the application event log, and various View and PCoIP logs), they can be time consuming to review, and often the information these logs provide is insufficient.

Using the Windows Performance Toolkit
The Windows Performance Toolkit is a set of tools provided in the Windows SDKs for both Windows 7 and Windows 8. It consists of two high level toolsets: A toolset to gather information, and a toolset to analyze information. Once users and systems have been found to have slow login times, the toolsets provided with the Windows Performance Toolkit can be employed to further ascertain what exactly is causing the slow logins.

Installation
This section details the installation process to get the tools on the system that is experiencing slow login times. This process assumes the use of the Windows 7 SDK. Below are the steps:

1. Remove Visual C# 2010 – this may or may not be necessary. If the C# version of the vSphere Client is installed on the workstation, then that existing installation of Visual C# 2010 will need to be removed. Not to worry, the SDK puts C# back on there, and there is no impact to the vSphere client or other applications that may use Visual C# 2010.
2. Install the Windows 7 SDK – this can be done HERE. Launch the winsdk_web.exe file and ensure that at least the Windows Performance Toolkit is selected, and then click Next. Once the installation has completed, move on to the next step.Note: In order to analyze Windows crash dumps (AKA BSOD) I keep the Debugging Tools for Windows installed as well.
3. Install .NET 4.0 – this can be done from HERE. Again, this depends upon whether or not it is installed on the workstation in question.

This completes the installation. The installation can be verified by confirming that the program group exists on the Start Menu, or navigating to the installation directory, which defaults to C:Program FilesMicrosoft Windows Performance Toolkit, and confirm the existence of xbootmgr.exe and xperf.exe as seen in the images below.

Using XPERF
The process to use XPERF to gather information regarding slow logins is as follows:

1. Enable fast user switching in the registry or GPO.
2. Create a local user account named Test, and add to the local administrators group. (Using an administrative user that is not the problematic user will also work.)
3. From the console of the problematic workstation, log in as the user with administrative privileges.
4. Launch a command line with elevated privileges, and navigate to C:Program FilesMicrosoft Windows Performance Toolkit.
5. Launch the XPERF command:
   1. XPERF Command: xperf -on base+latency+dispatcher+NetworkTrace+Registry+FileIO -stackWalk CSwitch+ReadyThread+ThreadCreate+Profile -BufferSize 128 -start UserTrace -on “Microsoft-Windows-Shell-Core+Microsoft-Windows-Wininit+Microsoft-Windows-Folder Redirection+Microsoft-Windows-User Profiles Service+Microsoft-Windows-GroupPolicy+Microsoft-Windows-Winlogon+Microsoft-Windows-Security-Kerberos+Microsoft-Windows-User Profiles General+e5ba83f6-07d0-46b1-8bc7-7e669a1d31dc+63b530f8-29c9-4880-a5b4-b8179096e7b8+2f07e2ee-15db-40f1-90ef-9d7ba282188a” -BufferSize 1024 -MinBuffers 64 -MaxBuffers 128 -MaxFile 1024
6. Using fast user switching, switch users, and login as the problematic user.
   1. Once the login has completed, stop the trace using the following command:
      xperf -stop UserTrace -d merged.etl
7. Gather the merged.etl trace file for analysis.

Using XBOOTMGR
In some cases, it may not be possible to switch users using fast user switching. In many cases, it may be easier to have the user run XBOOTMGR. This tool, when run, reboots the system and tracks both the startup time and the login time. The analysis ends after a set period of time. Gather an XBOOTMGR analysis by performing the following:

1. Launch a command line with elevated privileges, and navigate to C:Program FilesMicrosoft Windows Performance Toolkit.
2. Run the following command:
   1. XBOOTMGR Command: xbootmgr -trace boot -traceflags base+latency+dispatcher -stackwalk profile+cswitch+readythread -notraceflagsinfilename -postbootdelay 120
3. The system will prompt that it is being rebooted. Allow the reboot to occur.
4. When the VM is started, have the user connect to the View desktop using the View client.
5. When the user logs in, XBOOTMGR will present the user with a countdown of 120 seconds. Allow XBOOTMGR to collect data.
6. Once complete, gather the *.etl trace file for analysis. It may take some time to merge the file.

Analysis
The trace file has been created, and now it is time to analyze the results. The analysis toolset available in the Windows 7 Performance Toolkit is slightly different than what is available in the Windows 8 Performance Toolkit.

Performance Analyzer from Windows 7 Performance Toolkit

Open with Performance Analyzer (From the Windows 7 Performance Toolkit)

The graph below shows the processes occurring during the Winlogon Init process. It is easy to see that VMWVvpsvc is running for approximately two minutes.

By right clicking on the graph, one can Overlay Graphs from other categories. This graph shows the Winlogon process, as well as the overlay graphs for Boot Phases and CPU Usage. This can be helpful to see which boot phase the processes are running. Additionally, the CPU graph will show whether the process is running long because it has maxed out the available CPU capacity.

These overlays can be tweaked by selecting the CheckPoints box in the top right corner of the graph.

Windows Performance Analyzer from Windows 8 Performance Toolkit

Open with Performance Analyzer (From the Windows 8 Performance Toolkit).  The icon is shown below:

When looking at the same trace file as before, the graphs show that VMWVvpsvc was running for over 2 minutes. Moving the user files closer (from a network perspective) to the View desktop will help reduce the login time.

References
http://social.technet.microsoft.com/wiki/contents/articles/10128.tools-for-troubleshooting-slow-boots-and-slow-logons-sbsl.aspx

http://www.liquidwarelabs.com/products/stratusphere-ux

Gourav Bhardwaj is a VMware consulting architect who has created virtualized infrastructure designs across various verticals. He has assisted IT organizations of various Fortune 500 and Fortune 1000 companies, by creating designs and providing implementation oversight. His experience includes system architecture, analysis, solution design and implementation.

Matt Larson is an experienced, independent VMware consultant working in design, implementation and operation of VMware technologies. His interests lie in enterprise architecture related to datacenter and end user computing.