Home > Blogs > VMware Workstation Zealot


Using the VMware Visual Studio plugin

Last year for Workstation 6.0, one of the features we added was a plugin for Visual Studio 2005 that allows developers to debug a project (native and managed C/C++, C#, or Visual Basic) inside of a Windows virtual machine (specifically Windows 98, Windows 2000, and later).  All of you Windows developers know how much of a pain it is to support multiple versions of Windows, but this tool is designed to make that process much, much easier.  The Visual Studio Integrated Debugger (VSID in short) plugin utilizes Visual Studio remote debugging technology to allow you to debug a project inside of a VM as if you were debugging on your host computer with the click of one button.

The plugin’s toolbar and menu, added to Visual Studio 2005:

Toolbar_3
Menu_2

The most difficult part of getting all of this to work is the initial setup.  Once you can get your configuration working,
it should work well for a long time.  But it is certainly a chore getting your host and guest set up to comply with all of the security regulations that Microsoft requires to successfully remotely debug on another machine.  This walkthrough is intended to get you through the process of setting up remote debugging and provide helpful troubleshooting tips in case everything does not go smoothly.

For a full list of all setup steps, Chapter 2 in the VSID manual is a must-read (open up Visual Studio 2005 and go to the VMware menu and click Help Topics), but I will try and summarize the main points.  The first thing you will need to do is to make sure that you have the same username and password on your host and guest.  This is a built-in security requirement so not just anyone can remotely debug in your machine; Visual Studio automatically uses your host’s login and password as authentication and if they are different than the guest’s, it won’t work.  It is very strongly recommended that both your host and guest be on the same domain (or if not on a domain, that they both be in the same workgroup), although my own testing has shown me that it is not always necessary.  Anecdotally, I have found that Vista hosts are a little more strict about this than XP hosts, but it is certainly suggested if at all possible.

The next task is to configure the firewall on both your host and guest.  For the guest, we recommend that you disable the firewall.  For the host, I would recommend that at the least, you set Visual Studio as an exception so that the firewall leaves it alone.  If possible, I have found it helpful to disable the firewall on your host as well, because it can sometimes be a bit disruptive.  If you have any other security products running on your host or guest, you should make sure that they are also configured correctly.  For more information about setting up remote debugging, here are some instructions from Microsoft.  This is such a crucial step and I have found that most remote debugging failures come from a host firewall blocking remote debugging.

The final step in setting up remote debugging is changing a few settings in your guest.  Here is a short list:

  1. Disable the security prompts that come from running programs off a network share (which is what happens when the plugin runs your host program in the guest through Shared Folders).  To do this, open up Internet Explorer, choose Tools > Internet Options > Security > Local Intranet, click Sites, click Advanced and add a new Web site "file://*..host" (without the quotes).
  2. Make sure the name of your virtual machine is unique on the domain.  Conflicts like this often happen if virtual machines are cloned and handed around between multiple people.  To change it, go to Start > Control Panel > System > Computer Name and select Change.  On Windows 2000, the tab is named "Network Identification" instead of "Computer Name".
  3. For Windows XP virtual machines, change the guest user authentication scheme to Classic.  To do this, go to the Control Panel > Administrative Tools > Local Security Policy > Local Policies > Security Options page and set "Network access: Sharing and security model for local accounts" to "Classic – local users authenticated as themselves".  Windows XP Home does not have this setting and does not respect it even if you set the corresponding registry key, so that’s why we don’t support debugging in XP Home guests.
  4. If you are running a Windows 98 VM, you will need to perform a couple of extra steps.  Take a look at the VSID user manual for the list.

Now that you have your virtual machine set up for remote debugging, it’s time to set up your Visual Studio project.  Thankfully, this is much easier.  If the VSID plugin has been installed in Visual Studio 2005, you should see an extra toolbar and a VMware menu.  To configure your project, open up your solution in Visual Studio and go to the VMware menu and click Options.  To set up your project for remote debugging enter the following pieces of information:

  1. In the Command space, enter the full path to the executable that you want to debug in the guest.  If it is a path on your host machine, make sure "Run Command as" is set to "a host path through a shared folder".  If it is a path in your virtual machine, make sure that "Run Command as" is set to "a guest path".
  2. Verify that the Remote Debug Monitor path is set correctly.  If you are debugging a 32-bit application, it should be the path to the 32-bit remote debug monitor.  If you are debugging a 64-bit application, it should be the path to the 64-bit remote debug monitor.  Typically, the 32-bit remote debug monitor is installed in %Program Files%\Microsoft Visual Studio 8\Common7\IDE\Remote Debugger\x86\msvsmon.exe.
  3. Click "Virtual Machine" in the left column and enter the path to the virtual machine in the "Virtual Machine" space of that page.
  4. Look through any other options that you would like to specify.  However, all other settings are optional or already have a default specified.

One of the configuration pages:

Configpage_2

If all goes well, you should be able to start debugging in your virtual machine by hitting VMware > Start.  If you get an error message, the unfortunate situation is that the Visual Studio extensibility framework doesn’t give us any information about why debugging failed, so you are left to go through more official routes to get that information.

The first reason why debugging often fails is that your virtual machine is not configured to run your application properly.  If your application requires the .NET Framework to be installed, make sure that it is installed.  You should also make sure that all the dll’s you need are registered in the guest and any other configurations are all correct.  To verify that your application is configured properly, simply go to the VMware menu in Visual Studio and select Start Without Debugging.  If your application runs properly, then you don’t have any of these problems.  But often times you will see an error message complaining that a required component is missing.  This is probably the reason why remote debugging is failing, because Visual Studio is trying to debug the program in your guest but the program can’t even run successfully.

If you are still running into problems, the next step is to manually remotely debug in your virtual machine using Visual Studio’s built-in framework.  The reason why this is important is that Visual Studio will give you much more descriptive error messages than the extensibility framework gives to the plugin.  Here are the steps to set it up using a C/C++ project:

  1. With your virtual machine running, go to Workstation, open up the VM Settings editor, select the Options tab, and select Shared Folders.  Enable Shared Folders and add two shared folders: 1) Add the folder your application is being compiled in on your host machine; name it "Project". 2) Add the folder where the remote debug monitor is on your host machine; name it "RDM".
  2. Go into your guest and open up ..host\Shared Folders\RDM\msvsmon.exe.  This is the equivalent of running your host’s remote debug monitor in your guest.
  3. Open up the Project menu and select Properties.  Open up the Configuration Properties node on the left and select Debugging.  Now write down any existing debugging settings you have, because we may be overwriting these.
  4. Under "Debugger to launch:" select Remote Windows Debugger.  For "Remote Command", enter the path to your host executable using the Shared Folder you just created.  This will be \\.host\Shared Folders\Project\<executable name>.exe
  5. Change "Remote Server Name" to the name of the server that the remote debug monitor created in your guest.  To find this, look in your guest at the remote debug monitor; it should say something like "msvsmon started a new server named ‘grossag@foo’. Waiting for new connections."  In this case, the remote server name is grossag@foo
  6. Now go to Start > Debug.  If remotely debugging through the VMware plugin wasn’t working, then this should fail as well and give you a better error message.

If debugging fails with the following error, you should follow the troubleshooting tips above.

Error

Well that’s about it for me.  Good luck with remote debugging and feel free to post any questions that you have!  And I almost forgot that if you are looking for Visual Studio 2008 support, feel free to download our new Workstation 6.5 beta, although I will warn you that the current beta (build 84113) doesn’t work well with debugging in Windows 2000 guests.

14 thoughts on “Using the VMware Visual Studio plugin

  1. Peter

    Can you run Visual Studio in one virtual machine and remotely debug code running in _another_ VM? Reason: I’m on Linux and use VS inside a VM. I thought this would be impossible, but with all the talk about networking above, I’m beginning to think it may work (because why would you have to mess with network settings if it used some direct hostguest interface?)…

  2. Adam Gross

    You can definitely remotely debug from one VM to another VM, but you cannot use the plugin that we provide. The reason is that the plugin is set up to launch the VM in the machine where Visual Studio 2005 is installed (in this case, it’s a VM) and to do all of the debugging in there. So unless you could get nested VMs working in your configuration (only works on some processors, search “nested” in the Workstation forums), it wouldn’t work for you.
    That said, it’s really not too bad to just manually set up remote debugging as I explained towards the end of the article. The only difference would be that instead of sharing the remote debug monitor and application folder as shared folders in the VM, you would just have to share those two folders in your VM with Visual Studio 2005 through the network.
    Someone asked a similar question on the forums, so you can get more information here: http://communities.vmware.com/message/825906

  3. Herve Kabla

    Where can we find this plugin? Can(t find it in the WMware website.

  4. linh

    I have visual studio 2005 installed on the Windows 2000 host. How do a use this visual studio on the Windows XP guest just like I would on my win 2000 host?
    Thank you much.

  5. eu

    How about using the plugin from eclipse?

  6. Luke

    I couldn’t find anywhere to report a bug, so I’m going to do it here (hopefully someoe will read this). If you install the Visual Studio 2005 plugin and disable the VMware services, Visual Studio 2005 will hang at the splash screen while trying to load the plugin; you have to disable the plugin (hold down left-Shift) in order to run Visual Studio 2005.

  7. Adam Gross

    Luke, I believe you are referring to this issue: http://communities.vmware.com/message/1060630 . If so, please respond to the forum thread and tell them that you are running into the issue as well. If not, it would be best to open up a new thread in the VMware Workstation forums so we can get more detailed information from you. Thanks.

  8. Brian

    Hi, does the VS2008 plugin work on vmware server?
    I develop on win2k3 server.
    I tried to install workstation 6 and it blue-screened my box, so I had to go back to vmware server. (Too busy to debug your code as well as mine 🙂 )

  9. Adam Gross

    Brian, thanks for the comment. This plugin is only installed with VMware Workstation. It’s unfortunate to hear that you had problems with Workstation; maybe if you run into a less busy season you could try it again and check out the forums to see if there’s any help with that.
    Thanks,
    Adam

  10. Adam

    Hi, the VMware 6.5 installer only seems to have a VS 2005 Plugin component.
    I thought that VMware 6.5 was supposed to have VS 2008 support, as it was mentioned at the end of the article as well.
    Thanks

  11. Adam

    Ahhh, right. It installs for VS 2008 as well. It’s only the installer’s comment which is misleading for it only mentions VS 2005.

  12. Adam Gross

    Hey Adam, thanks for the comments. I will file a bug internally to get this fixed.

  13. Christian

    I am trying to debug an application that accesses the registry and I’m getting an:
    Request for the permission of type System.Security.Permission.RegistryPermission
    I’ve added file://*..host to my trusted sites, but it seems like I need to give it a bit more to do some of these features.
    Any ideas as to how to address this?
    Thanks.

  14. Clint

    Just a tip:
    Where it says “user name and password must match host machine”… That’s not enough.
    It really means “User name on VM must have *been created* the same as on the host machine”
    If you create a user name of “Clinton” and later change the name to “Clint” to match the host… All will look right everywhere, yet VS will never be able to log into the VM guest.
    Despite renaming the user account to match, somewhere it is retaining the account creation name.
    I tore my hair out for 3 days over this.

Comments are closed.