Scaling a vRA 7.3 Environment (Part 2)

Last time we installed a distributed vRA 7.3 environment by using only one node per role.

Here comes the time when you have to add another node for each role, to the vRA setup, because vRA has become a critical asset and you want to lower the downtime of the service.

There are a few ways to do it and most people usually choose the most boring one – using VAMI and the Suite Installer (meh). You’ve got other options, though. Options like the uber cool vra-command tool present in every vRA virtual appliance instance. Apart from acting like the cool kids on the block, automating these tasks gives you the option to add nodes whenever you feel the need for better performance of your cloud services.

Appliance

Let’s begin with adding a second virtual appliance to the environment. The process is a bit weird – we’re not really adding a second appliance, but inviting it to join the cluster on behalf of the current node. For clarity, let’s just name the already installed node Node01 and the new node Node02. This is the procedure you have to follow:

1. Deploy the new appliance with all needed host settings like IP address, DNS servers, etc.
2. Connect to SSH on Node01.
3. Get the VAMI certificate of Node02, so Node01 can trust it:

NEWVACERT=`echo -n | openssl s_client -connect node02.domain.local:5480 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'`

4. Get the node ID of Node01:

vra-command list-nodes

5. And now issue the following command:

vra-command execute --node cafe.node.ID cluster-invite --VaHost node02.domain.local:5480 --VaUser root -–VaPassword SomePass  --VamiCertificate "$NEWVACERT"

Just make sure to substitute the node ID and password parameters.

And you’re ready to proceed with adding the two servers to your load balancer of choice. Don’t use DNS round robin – it’s not even a poor man’s solution to load balancing.

Manager

How about adding a new manager server, so you can be sure it’s always on? vRA 7.3 features automatic failover of the Manager service which is one of the best new additions to the product.

1. Install your new Manager server and all needed prerequisites.
2. Import the Manager service certificate into the Machine Personal certificate repository.
3. Install the Management Agent from https://vanode:5480/i. Since we’re on the topic of automating vRA, you can just use a script to install the agent.
4. Connect to SSH on Node01.
5. Get the Node ID of the new Manager server:

vra-command list-nodes

6. Get the fingerprint of the Manager service certificate (make sure you connect to the Load Balancing FQDN that you specified during installation):

MGRCERT=`echo -n |openssl s_client -connect  nn-scale-mgr.domain.local:443 | openssl x509 -noout -fingerprint|sed -e 's/://g' -e 's/^.*=//'`

7. Install the service:

vra-command execute --node "Manager-Node-ID" install-manager-service --SqlServer "your-sql-server-fqdn" --DatabaseName "your-db" --UseWindowsAuthentication True  --IaaSWebAddress 'nn-scale-web.domain.local' –SecurityPassphrase 'dbpass' --ServiceUser "domain\\nn-svc-vcac" --ServiceUserPassword 'ServiceUsrPass' --ManagerServiceStartAutomatically True --ManagerServiceFailoverModeEnabled True --ManagerServiceCertificate "$MGRCERT" --VraAddress "nn-scale-va.domain.local"

Make sure to substitute all parameters according to your environment. Did you see the ManagerServiceFailoverModeEnabled flag? It is mandatory if you want your new manager server to assume the passive role automatically.

8. Configure your load balancer.

DEM

The procedure to add a DEM orchestrators and Workers is almost identical. Just issue the following commands:

1. Get the node ID of the node you’re trying to install. You should have the management agent running already on it.

vra-command list-nodes

2. Get the vRA Automation Console certificate fingerprint (the load balancing endpoint of the VAs):

VRACERT=`echo -n | openssl s_client -connect  nn-scale-va.domain.local:443 | openssl x509 -noout -fingerprint|sed -e 's/://g' -e 's/^.*=//'`

3. Install the DEM Orchestrator:

vra-command execute --node "DEMOnodeID" install-dem --ServiceUser "domain\\nn-svc-vcac" –ServiceUserPassword 'ServiceUsrPass' --DemName "DEM2" --DemDescription "Secondary DEM Orch" --DemRole Orchestrator --ManagerServiceAddress "nn-scale-mgr.domain.local" --IaaSWebAddress 'nn-scale-web.domain.local' --WebUserName "domain.local\\nn-svc-vcac" --WebUserPassword 'WebUserPass' --VraAddress "nn-scale-va.domain.local" --VraWebCertificateThumbprint "$VRACERT"

The WebUserName parameter in most cases is the service account username you’re using.

4. Install the DEM Worker:

/usr/sbin/vra-command execute --node "DEMWnodeID" install-dem --ServiceUser "domain\\nn-svc-vcac" --ServiceUserPassword 'ServiceUsrPass' --DemName "DEMW2" --DemDescription "Second DEM W" --DemRole Worker --ManagerServiceAddress "nn-scale-mgr.domain.local" --IaaSWebAddress "nn-scale-web.domain.local" --WebUserName "domain\\nn-svc-vcac" --WebUserPassword 'ServiceUsrPass' --VraAddress "nn-scale-va.domain.local" --VraWebCertificateThumbprint "$VRACERT"

No load balancing configuration is needed for the DEM roles.

Web Certificates

Finally, the Web Server installation seems like the most difficult part to implement, because of the certificate issues. Remember how we installed the Web role by using a SAN certificate with only one Web node? If we want to successfully install a new web server we should first replace the existing certificate with one containing the new node’s FQDN:

1. Create a new certificate with the new node’s FQDN:
2. Convert the certificate to a PEM file containing both the private key and the public key chain.
3. Set the public and private keys to some variables, e.g. $publicKey and $privateKey. You can use your preferred tool for extracting them.
4. Get your current Web Server Node ID:

vra-command list-nodes

5. Install the certificate:

vra-command execute --node "WebNodeID" install-certificate --CertificateData "${publicKey}" --PrivateKeyData "${privateKey}" --CertificatePassword "CertPass" --CertificateFriendlyName "new scaled web service" --StoreNames "My;TrustedPeople" --StoreLocation "LocalMachine"

This command will automatically import the new certificate int the certificate store of the specific node. The downside is that it should be executed against each web server if you already have more than one and also you will have to actually manually configure your IIS. Therefore, for now my recommendation is to just use VAMI or the API (coming soon as part three) for certificate replacement.

Web

Finally, we’ve got everything in place, so let’s just go on with installing our new Web server:

1. Install the Management agent.
2. Get your new Node ID.
3. Import the Certificate in the Personal store of the machine.
4. Get the Web certificate thumbprint (hint: you can modify one of the above openssl commands).
5. Get the vRA Automation Console certificate thumbprint.
6. Install the Web role:

vra-command execute --node "WebNodeID" install-web --SqlServer "SQLServerFQDN" --DatabaseName "nn-scale-vra" --UseWindowsAuthentication True  --IaaSWebAddress 'nn-scale-web.domain.local' --SecurityPassphrase 'SecurityPass' --ServiceUser "domain\\nn-svc-vcac" --ServiceUserPassword 'ServiceUserPass'  --VraAddress "nn-scale-va.domain.local" –VidmAdminPassword 'vIDMAdminPass' --VraWebCertificateThumbprint "vRACertThumbprint" --WebCertificate "WebCertThumbprint"

So, this is it. Using vra-command is a great way to manage your vRA infrastructure. Go ahead and play with its other options.