From the Trenches Management

Troubleshooting when virtual machine operations are greyed out

Hello, I am a Technical Support Engineer at VMware. I would like to talk about an issue I worked on recently with a customer that may come up for other users. The Support Request (SR) came to me with this description: “The Virtual Machine options are not working”. I had to ask myself, “What do you mean by ‘not working’”? I quickly called the customer and we got a remote session going so that I could see for myself what the problem was.

The problem was with only one Virtual Machine in that vCenter instance. I found that the virtual machine options were disabled (greyed-out) when I right-clicked the problematic virtual machine.

Note: Click an image to see the full-sized version.

The customer at this point mentioned that he had tried to take a backup using third-party software. He saw this error message in the “Recent Tasks” pane:

My initial hunch was that there was a job running for the virtual machine in the background, but I needed to verify it wasn’t a permissions problem.

Permissions are not set correctly

First I checked the permissions given to that virtual machine. The customer had a very small environment, so I did not spend too much time checking the complete permissions given for the entire vSphere environment. Instead, I went through the following steps:

Note: If your environment is large and you have multiple vSphere administrators with different permissions, permissions on a particular virtual machine machine might be incorrect/missing. In this case, the vSphere root administrator needs to ensure that there are sufficient permissions given for you to administer the virtual machine, at all levels.

  1. Check the Permissions tab for the virtual machine, to make sure that your name is listed there with the proper permission assigned. If your name is not there, but an AD/Local group is listed, then make sure that your name is added to the AD/Local group.Here, the user Testuser has the Virtual Machine Administrator role:
  2. If the permissions are defined at the host, cluster, Datacenter, or vCenter level:
      1. Apply the required permissions to the user/group.
      2. Make sure that the permission is propagated to all the objects.
      3. Make sure that your name is listed in the local/AD group.Here, the group Virtual Machine Administrator has the Virtual Machine Administrator role:

  3. Roles can also be assigned in the folder level. Go to the VMs and Templates view and make sure the folder where the Virtual Machine located has all required privileges assigned.In the below example, VM1,VM2 and VM3 are located in the folder Test. The Virtual Machine Administrator group is assigned with “Virtual Machine Administrator” role.
    In the examples above, the role “Virtual Machine Administrator” is not a default role, it is created with privileges to Administrate Virtual Machines. To know what privileges are required, I always refer to the Required Privileges for Common Tasks section in: Virtual Machine Administration Guide.

With that set of checks, we can now go on to the next step, and this comes back to my hunch because of the error message which I noticed in the “Recent Task”.

Tasks Running in the Background

We should always be aware that certain jobs for any given Virtual Machine may be running in the background which will not allow any changes/operations in the Virtual Machine. These background jobs cannot be seen in the “Task and Event” tab in the vCenter server or in the “Recent Tasks”.

The method we look for these background jobs is by logging into the host via SSH where the problem Virtual Machine is running. Here are the steps to find any running jobs which cannot be seen in the vSphere client.

1. Log into the ESX/ESXi host using a SSH client.
2. Run the following command to list all the Virtual Machines registered in the host. We are running this specifically to find the vmid for the problem Virtual Machine.

The output appears as:

3. Check the tasks running on the Virtual Machine by running the command:

In our example, we run the command against the Virtual Machine VM3. It indicates there are 2 jobs/tasks running on that Virtual Machine.

The output shows as below if there is no jobs/tasks running.

4. Now you you have a choice to either wait for the jobs to complete, or restart the Management agent to terminate this job.

In this particular support case I was working we found a snapshot job that was triggered by the backup agent running in the Virtual Machine. I stopped the job by restarting the management agent on that host. For this customer, this solved the problem, but there is one more reason this might happen. I had a word with one my colleagues, and he pointed out that you could also encounter this if you have an invalid entry in your VMX file. Let’s go one step further and show you how to check this.

Invalid entries in VMX file

This type of issue might happen if a vmx file has invalid parameters or blank lines in it. You can resolve this issue by manually removing the invalid arguments or deleting the blank lines.

Caution: If incorrectly done, your virtual machine may fail to start or operate incorrectly. Always take a backup of your .vmx file before modifying it.

  1. Open the .vmx file using any text editor.
  2. Search for any blank lines and delete them.
    Note
    : To delete a single line using vi editor, press d twice.
  3. Compare the .vmx file with a working virtual machine .vmx file and see if there are any invalid arguments.
  4. To apply the changes, reload the .vmx file by running the command:

Note: The Default location of the vmx file is:

Vmfs/volumes/name_of_the_datastore/vm_name/vm_name.vmx

Summary

Symptom:
Virtual machine operations are grayed out

Possible Causes:

  • Permissions are not set correctly
  • Tasks Running in the Background
  • vmx file is corrupted

For further information on the steps used in this article, refer to: Troubleshooting when virtual machine options are grayed out in vSphere Client (2048748)

Hope this blog post helps you out a little bit. Have a great day!

Comments

0 comments have been added so far

Leave a Reply

Your email address will not be published. Required fields are marked *