vSAN Support Insight is a next-generation platform for analytics of health, configuration, and performance telemetry. It’s purpose is to help vSAN users maintain a reliable and consistent computing, storage and network environment.
How it works
It is a phone home system leveraging the existing vCenter Customer Experience Improvement Program (CEIP). Health, performance, and configuration information for VMware Support and engineering. Once enabled and verified, log information is pushed back to VMware’s Analytics Cloud on a regular cadence. No actual customer data is phoned home and user enterable or identifiable data (Hostnames, VM names, subnets, IP addresses, Mac addresses) are obfuscated.
How this benefits VMware customers
Immediate awareness of the state, configuration, and history of an environment delivers faster time to resolution on support tickets. Avoiding the need for initial log collection should lower the operational requirements for running a VMware powered HCI environment.
Engineering is looking to identify and fix bottlenecks, and issues before they impact customers. Problematic software, hardware issues can be identified at scale through correlation of faults between multiple customers with similar configurations. The vSAN team is working with VMware Skyline to use Support Insight for shifting Premier support services to a less reactive support experience.
vSAN online health checks today allow VMware to push out new recommendations that have been identified to be high impact for vSAN customers. The addition of vSAN Support Insight provides the ability for engineering to model the impact of new vSAN Online Health Checks across customer environments, and GSS to test them before they roll out to customers. Testing against the phone home data allows for refining of tighter thresholds to avoid false positives and negatives as well as the ability to measure the scale and impact of an issue.
As of vSAN 6.7 Update 3 vSAN Support Insight information, status, and configuration is displayed at the following locations
vCenter Level
- Day 0: During the initial configuration of a vCenter Server
- Day 2: In vCenter under Menu > Administrations > Deployment > Customer Experience Improvement Program
Cluster Level
- Day 1&2: Running the Cluster Quickstart wizard in vCenter, from the Cluster > Configure > Configuration > Quickstart location
- Day 2: Click on Configure > vSAN > Services > Support Insight (will only show up if disabled)
- Day 2: Click on Monitor > vSAN > Health > Online Health Availability > Customer Experience Improvement Program
How this is being used for support
vSAN Support Insight is for customers with production support, this is being integrated for reactive support purposes. All vSAN customers with an active support contract running vSphere 6.5 Patch 2 or later. CEIP data is transferred using SSL (HTTPS) from the vCenter Server. This will leverage a proxy if configured for the vCenter server. For information on troubleshooting the network connectivity see this guide. It is specifically sent to https://vcsa.vmware.com . For specific firewall and network information, please see the following guide. vSAN is currently using CEIP data for Configuration, Health, and performance telemetry. Information with unique information is obfuscated. For more information on how to view the data sent home see this guide of how to export the JSON files. For more information on the CEIP program see the CEIP FAQ. Also, the CEIP homepage has more information regarding what is being collected and its purpose. No VMs, VM names, IP address’s, Host names or any user configured information is communicated to VMware as a part of its analytics gathering. It is all anonymized.
CEIP configuration and information
Customers often want to know what a phone home system is sending back. CEIP is anonymized. Find a sample export here as well as information on how to configure, and export it.
Prerequisites
- vSAN Online Health needs to connect to “https://vcsa.vmware.com” and “http://www.vmware.com” to perform online health checks.
- If the system uses a firewall or a proxy to connect to the internet, then a firewall/proxy rule allowing outbound traffic through for “https://vcsa.vmware.com:443/ph/api/*” is required
Note: vCenter 6.5U1 has a known issue with CEIP failing when a proxy is in use. This will be resolved in a future patch, but a workaround is available from support.
Step 1: Check CEIP status
In the vSAN Health checks, a CEIP check shows the CEIP status. If CEIP is disabled, click ‘Configure CEIP’ button or ‘Enable Online Health’ Button, which will navigate to the CEIP page to opt in.
Step 2: Trigger the first upload
Once you have enabled CEIP, click the ‘Retest’ Button. A new check appears in Online health called “Online health connectivity”, and the ‘Enable Online health’ button changes to ‘Retest with Online health’ . click this button to trigger the first upload.
Step 3: Check the Online health work properly
Once the ‘Retest with Online health’ button is clicked, this button changes to gray, and an online health check task is triggered.
Once the online health check task is done, the task status is ‘Completed’, and ‘Retest with Online health’ button is back to normal, and new health checks appear in the Online health group with the group name updated last check time.
Troubleshooting
If connectivity is failing, you can SSH into the vCenter and try to communicate with the CEIP webserver using the following command.
curl -v https://vcsa.vmware.com:443 “
If you see “fail to connect”, further tests of dns or connectivity may be needed.
To test DNS connectivity use the following command.
“nslookup vcsa.vmware.com “
Export of CEIP
Online Health checks collect data from the environment and send it to VMware cloud to perform health checks. The information below helps you understand how to inspect what telemetry data a live environment is sending. You can check the data collected as a part of health checks as follows:
Linux Environment:
- SSH into VCSA
- Run command: cd /var/log/vmware/vsan-health/
- Data collected by online health checks is written and gzipped to files ” cloud_health_check_data.json.gz” and ” vsan_perf_data.json.gz
- You can extract the json content by calling ” gunzip -k ” or view the contents by calling ” zcat “
Windows Environment:
- Login to Windows vCenter machine.
- Open ” C:\Program Files\VMware\vCenter Server\logs\vsan-health “
- Data collected by online health checks is written and gzipped to files ” cloud_health_check_data.json.gz ” and ” vsan_perf_data.json.gz “
- You can use 7zip or any other utilities to unzip the gzipped files
Data collected is obfuscated to comply with CEIP. All private data is obfuscated. You can find the obfuscated data mappings in files _obfuscationTableForHuman.json.gz .
Note: The obfuscation mapping files are not sent to VMware. Only obfuscated data is sent to VMware.
