Part of my job as a Customer Success Architect is to consult customers on various topics around vRA. Sometimes, those topics go to uncharted territory. One of them is managing the IaaS service account and its password according to security protocols. VMware supports this and we even have a procedure defined in a KB, but once we throw it at the customer we immediately get a bunch of additional questions around it. How to automate it? What happens to provisioning requests during the password change? How do we verify it’s working? To all these questions, I actually have a single answer – Group Managed Service Accounts.
I decided to make a good and tested procedure for configuring Group Managed Service Accounts for vRA IaaS services. GMSAs are not a very popular feature of Windows, but I think they come very handy for various reasons. By default, their passwords change automatically every 30 days, they are very complex and nobody can change the password manually. Oh, and the best feature – services don’t need a restart after a password change. The procedure below is tested on both vRA 7.2 with Windows Server 2012 R2 and vRA 7.3 with Windows Server 2016.
You might think it all starts with the installation, but it actually all starts after installing the product. During installation you just use any ordinary Active Directory service account with required permissions:
Why not install it with the GMSA directly? Well, while the password of the GMSA can be decoded easily if you have enough permissions, the cleartext representation is a combination of 128 UTF-16 characters (i.e. lots of strange characters and hieroglyphs) which are not well accepted by IaaS installers. Only after you have successfully installed your vRA solution, do you go and configure the Managed Service Account. However, have no fear – subsequent maintenance and upgrades work flawlessly.
I have based the procedure and respective commands below on a blog from Microsoft Technet and also followed VMware’s KB article on manually changing a service account. The procedure assumes you will be using the same Service Account for all IaaS services as well as the Management Agent.
In order to run the commands, you need to be a member of Domain Admins and have the ADDS PowerShell module installed.
So, you start with defining some parameters like the IaaS hostnames (no FQDNs needed), the account name, the IaaS web load balancing FQDN (could also be the manager service load balancing endpoint, but one or the other), and SPNs needed for the account:
$hosts = 'nnvraweb1', 'nnvraweb2', 'nnvramgr1', 'nnvramgr2' $gMSAName = "NN-vRAIaaS-GMSA" $gMSAFQDN = 'nnvraweb.domain.local' $gMSASPNs = 'https/nnvraweb', 'https/nnvraweb.domain.local', 'https/nnvramgr', 'https/nnvramgr.domain.local'
Then, you have to create a KDS Root Key if one does not exist yet.
Add-KDSRootKey -EffectiveTime (Get-Date).AddHours(-10) #effective time is 10 hours ago because we work around a safety measure
A Group Managed Service Account is called like this, because a group of hosts can actually use it as opposed to the Managed Service Accounts in Windows Server 2008 R2 which did not allow configuring the same account for more than one server. You can either specify all hosts for vRA IaaS one by one or just create a new Active Directory group which makes it easier for you to add new servers whenever you want to scale out without re-configuring the service account.
$hostsGroupName = "NN-vRAHosts-GMSA" $hostsGroup = New-ADGroup -Name $hostsGroupName -GroupScope Global -PassThru -Path "OU=nikolovn,OU=Computers,OU=CustomerSuccess,DC=domain,DC=local"
$hosts | ForEach-Object { Get-ADComputer -Identity $_ } | ForEach-Object { Add-ADGroupMember -Identity $hostsGroupName -Members $_ }
Now, all you have to do is create the account with all the parameters you have already defined. You can see that we make it as secure as possible by allowing only the servers that are members of the group to retrieve the account’s password (which, if you haven’t noticed yet, is very, very secure). The parameter called ManagedPasswordIntervalInDays can only be specified at creation time, so if you don’t add it to the New-ADServiceAccount command, you’ll configure its password to expire every 30 days which is the default option.
New-ADServiceAccount -Name $gMSAName -DNSHostName $gMSAFQDN -PrincipalsAllowedToRetrieveManagedPassword $hostsGroup -ServicePrincipalNames $gMSASPNs -ManagedPasswordIntervalInDays 60
Since you created a new AD group, the IaaS servers should be made aware that they are members, so you need to restart them. If you don’t do it, you’ll get an Access Denied message when trying to install the GMSA on the operating systems.
Of course, you can implement the commands in this MSDN blog if reboot is not an option and you really want to go the hacker’s way.
After reboot, I recommend you to run “Install-ADService” on each Windows machine which will make it aware of the managed account and its password.
$gMSAName = "NN-vRAIaaS-GMSA" Add-WindowsFeature -Name RSAT-AD-PowerShell Install-ADServiceAccount -Identity $gMSAName
Next, add the user as an owner for the IaaS SQL Server database so it can connect to it. When searching for the user, make sure to select Service Accounts in the Object Types dialog box.
Add the user as a local administrator on the IaaS servers. Mind the dollar ($) sign at the end of the account name – MSAs are always represented with a $, because they are special type of accounts:
NET LOCALGROUP "Administrators" $gMSAName'$' /add
Change the service account for IaaS Web Application pools by right-clicking the pool and selecting Advanced Settings. Again, don’t forget to type the account name with the dollar sign at the end, but don’t specify a password.
Do an IISreset on all Web servers and then log in to the vRA console and check if the Infrastructure tab loads. Also, go to IaaS-Install-Dir\Server\Model Manager Web\logs and check the Repository.log for any errors. Also check the Event Viewer of the Windows machine for any errors.
Proceed with the Manager server, DEM orchestrators, DEM workers and Proxy agents by changing their service accounts and restarting the respective services. Do not specify a password there either. If you want to remain with the same Manager Server having the Active role, implement the changes on it first and then proceed with the second one. After the second one stops, the first one should assume the Active role again. Bear in mind that the mechanism of Manager Server failover works only after a certain period of time, so if you restart the services too fast, the failover might not happen.
Check the logs for any errors and also go to the vRA Console and select Infrastructure tab -> Monitoring -> DEM Status.
Finally, change the service account for the management agent. In order to check if the agent works fine, go to VAMI -> Cluster tab and check the Last connected status.
Sweet, you have configured vRA to work with a Group Managed Service Account!
Go and have a try at vRA!
