F5 load balancer common misconfigurations for vRA 7 Distributed setup

Working with several customers to build vRealize Automation 7.x in production environment has exposed me to many vRA Distributed installs that involved the use of Load Balancers. There seems common mistakes that customers fall for when configuring load balancers for vRA 7.x distributed environments that cause the installation to fail or the setup not to function as intended. Here I wanted to highlight few of these. While I will be focusing on F5 in here, these mistakes can affect other load balancers as well.

vRealize Automation 7 F5 load balancer

1- Utilize the load balancer VIP for initial installation

Several customers try to use their load balancer VIP during vRA installation. While if setup perfectly this will work, a small mistake with the VIP configuration can make the installation and configuration of vRealize Automation feel impossible. For this I would recommend you create the VIP DNS record and just point it to your first nodes. Complete your vRA installation and configuration and only after confirming your setup is stable and fully installed to point your VIP DNS record to your actual VIP IP. This will make your installation go much smoother, and allow you a much easier path to troubleshooting if you made a mistake during load balancer configuration.… Read More

vRA 7.2 what’s new?

vRealize Automation 7.2 which was promised at VMworld to enhance ease of use with Out-of-The-Box support for ServiceNow, Azure, and Containers. has just been released today as promised in VMworld. You can download it at: vRealization Automation 7.2 download link. This article highlight few of the major improvements introduced with vRA 7.2.

Azure Endpoint

Microsoft Azure vRA Endpoint

While Amazon AWS, vCloud Air and vCloud Air Network endpoints have been out of the box in vRealize Automation for the longest time, Microsoft Azure has always required custom coding integration to work in vRealize Automation. Many customers showed interest in being able to deploy Microsoft Azure and were waiting patiently for vRA to offer it out of the box. This is coming your way out of the box in vRA 7.2. This will extend vRealize Automation Hybrid Cloud offering reach to Microsoft Azure customers.

The new Azure endpoint will allows customers to:

  • Configure Azure connections (endpoints) per tenant
  • Assign reservations and integrate with their governance model
  • Design blueprints creation with Azure resources, specify network and storage options
  • Supports Azure Networking (subnets, load balancers)
  • Deploy converged blueprints with Azure resources in them
  • vRealize Automation can automatically select the most appropriate subscriptions to deploy to
  • Perform state-aware resource actions on their Azure resources
  • Incorporate key vRA functionality, such as approvals, extensibility, and CBP
  • OTB Day2 lifecycle operations include start, stop, restart, delete

Note: Azure Endpoint is the first to be built on top of extensibility platform (XaaS vs .Net) levarging the new Azure vRO plugin (Included OOTB).… Read More

vRealize Automation 7.x F5 Load balancer Configuration

This week I had to work with a customer who used F5 to load balance their vRA Distributed environment, and it has proven that just a small change from recommended configuration can break your full setup. For this reason, I have decided to document the full F5 load balancer configuration on this page. This post can still be helpfull to get you started with other load balancers as well.

As medium distributed install seems to be the most common between customers, I will use it as the base of my post in here, but I will cover the differences in large distributed install as well. Let’s start at first with the list of our setup components. As you will notice in the below table that the manager service and Web are both sharing the same servers unlike large architecture installation where Web and Manager Service are separated into different servers. That has been said, you will still need to setup a different VIP one for Manager Service and one for Web in both medium and large distributed setups. The main difference is that in a medium setup both VIPs will point up to the same servers. You will need as well to setup another VIP for the vRA Appliances.… Read More

vRealize Automation Order of Precedence for Custom Properties.

If you have just started working with vRealize Automation, you will notice that you can apply custom properties to many elements. While this is great and add a lot of flexibility to the product and make it easier to customize the product to do things differently in the way you desire. It can get confusing which property will precede/over write the other one, if a duplicate property with a different value was assigned at a different element/level. In this post, I wanted to highlight the order of Precedenc for Custom Properties in vRA, so you can predict your results.

vRA Custom Properties Order

The full order of precedence for custom properties is that any property value specified in a source later in the list overrides values for the same property specified in sources earlier in the list. The order is shown in the following list:

Order of applying Custom Properties, later one over write earlier one:

  1. Build Profile
  2. Blueprint
  3. Business Group
  4. Compute resource
  5. Reservation
  6. Endpoint
  7. Runtime

Note: Please note only Build Profile, Blueprint, and Business Group custom properties are assigned at the request time, while most other custom properties (Comput resources, reservation, & Endpoint) are assigned during provisioning, as they are not being assigned till the provisioning process has been started.… Read More

vCloud Air DR to Cloud with vRealize Automation

Traditionally organizations have reserved their Disaster Recovery capabilities to their most critical workload leaving some important workloads without proper disaster recovery due to the high cost of disaster recovery.

Imagine if you went shopping to buy a car with a friend. When you hit the dealership, your friend tell you that you should buy two cars. The first one will be the one you will be using, where the second one is only for you to use if your first car break? Do you find that as an odd advice or at least too costy one? Was not that exactly what you are doing when building your Disaster Recovery Datacenter? You build a full datacenter and fully pay for it and its compute, storage, cooling, space, and operation 24/7.  Luckily, you did not have a disaster this year, but does not that mean you just paid tons of cash for compute, etc that you never used?

What if you can get the same protection, but almost at the cost of only your storage? That’s what DR to Cloud allow you to do. In fact, vCloud Air DR to Cloud offering is great at that, as the cost you are normally paying for storage covers the cost of protecting your VMs in vCloud Air Disaster Recovery.… Read More

Replacing Certificates in vRealize Automation 7

If you worked with vRA 6.x and earlier, you will definetly know that replacing certs was a very involving process. The great news is that vRA 7 has eliminated most of the work required to do so and automated a good part of the process. Unfortunately, this is not clearly documented in vRA 7 documentation, although in my opinion this is one of the major improvements in vRA 7.0. As I have discovered those improvement the hard way through going through engineering and support, I thought It’s worth sharing as it can save you tons of time.

Let’s start by the scenario where you have setup the vRealize Automation 7.x system up and running, and you want to replace your certificates. Let’s assume you have already generated the certs for the different services, but if you did not and need help with that keep reading as I will give you guidance on that later on in this article. Below is how to replace the certs of each of the services making up your vRA 7.0.

– SSO/Identity Appliance: As vRA 7.0 does not make use of vCenter SSO any more and it moved to embeded vIDM, you will not have to worry about the certs of this any more.… Read More

vRealize Automation 7 Bootstrap Installation

vRealize Automation 7 converged blueprint is now offering you the capabilities to go beyond VMs deployment to being able to design/deploy a full application stack. In order to enable application deployment in vRA, you will need to install vRA 7 bootstrap in your VM template. In this post, I will cover how to install vRA 7.0 bootstrap into your Windows & Linux VM templates.

vRA 7 converged blueprint

Note: vRA 7.0 enterprise license is required to use the software deployment capabilities of vRA 7.0. If you have a license that does not include this, you will need to depend solely on the Gugent to run your scripts in similar fashion to what was done in previous versions of vRA, which I have document previously at: vCAC Guest Agent Installation

Note: In vRealize Automation 7.1 and higher the download link of the Guest Agent, Bootstrap Agent and so on has changed from the links used in the below article to https://vRealize_VA_Hostname_fqdn/software/index.html. 

vRA 7 bootstrap installation in Windows:

(Certs)

  1. Make sure your root cert is trusted by your template.
  2. Make sure your Manager Service cert is trusted by your template as well.

(Gugent)

  1. Download the Gugent installer from https://vRealize_VA_Hostname_fqdn:5480/installer/GuestAgentInstaller_x64.exe. The download link has changed to https://vRealize_VA_Hostname_fqdn/software/index.html in vRealize Automation 7.1 and higher.
Read More

VMware Hybrid Cloud Manager vs vCloud Connector

While some readers might be surprised of me comparing Hybrid Cloud Manager and vCloud Connector as many might assume that Hybrid Cloud Manager is replacing vCloud Connector, so why compare them. While this might be true in the long term, its still not the case today. While Hybrid Cloud Manager Excel at some of the features vCloud Connector used to deliver and add few that vCloud connector did not have, it still lack some of the features vCloud Connector had that you might still need. I have put this post to help you decided which one fit your need better or do you need both. Yes, there will be cases where you will need both products to co-exist.

Let’s compare Hybrid Cloud Manager and vCloud Connector on their merits in the different use cases:

Hybrid Cloud Manager vs vCloud Connector

Target Cloud it support:

  • Hybrid Cloud Manager is only meant to support VMware vCloud Air (Previously known by VMware Hybrid Cloud Service) Dedicated Cloud at the momement, where it will expand to the other vCloud Air offering in the near future. This might even expand to VMware vCloud Air Network providers in the future, but its not there yet.
  • vCloud Connector is currently supported with VMware vCloud Air, vCloud Director, and most VMware vCloud Air Network providers.
Read More

How to Pass Encrypted Custom Properties to vRealize Automation Guest Agent

Have you ever needed to pass an encrypted custom property to your vRealize Automation (previously known as vCloud Automation Center) gugent? I have been asked for this frequently by customers and below are just two of the most common use cases for this.

1- The customer wanted the Gugent to run a script that map a share drive to the deployed VM using the Net Use command where they need a different user credential to login to the map drive.

This is easy to do by letting the Gugent run a command similar to: “net use s: \\tower\movies /user HTG Pa$$word”. The challenge here is who wants to pass his password in clear text or store it in clear text in a script? Yop, I have deployed vRA for banks and government customers and none of them can accept that.

2- The customer want to be able to join the VM to one of many domains. They wants to achieve this without having to create tons of Custom Specifications in each vCenter they have as that will be a management nightmare in the long run.

This as well can be easily achievable by having a script as the following:

==================================

REM JoinDomain.bat script start here

If %1==firstdomain netdom join %%computername%% /domain:firstdomain.COM /userd:%2 /passwordd:%3
If %1==2nddomain netdom join %%computername%% /domain:2nddomain.COM /userd:%4 /passwordd:%5
If %1==3rdomain netdom join %%computername%% /domain:3rddomain.COM /userd:%6 /passwordd:%7

REM JoinDomain.bat script End here

==================================

You will then run the above script from vRA using a command like: JoinDomain.bat {domainname} {firstdomainUser} {firstdomainPassword} {2nddomainUser} {2nddomainPassword} {3rddomainUser} {3rddomainPassword}

Again the above resolve the problem, but you will need to pass the password in clear text.… Read More

VMware Cross-Cloud vMotion is another major milestone on VMware Seemless Hybrid Cloud Vision

If you are like me or many of VMware customers who has been following the VMware Hybrid Cloud Story closely for a while, you have probably asked yourself or your sales rep when will you be able to move your Virtual Machines back and forth between On Premise (Private Cloud) to vCloud Air (VMware Public Cloud) without encountering downtime. Many of us are used to the concept of VMotion and being able to move our VMs across hosts within a site and in vSphere 6 we had the capability to move VMs across sites and vCenters.

The next logical step in that vMotion evolution is being able to vMotion VMs to the Cloud and back, and that is just what VMware has announced at VMworld today. In fact, a live demonstration of what is called Cross-Cloud vMotion was shown during today VMworld General session. This exciting news has been a part of  Technology Preview of Project SkyScraper, a new set of hybrid cloud capabilities for VMware vSphere that will enable customers to confidently extend their data center to the public cloud and vice-a-versa by seamlessly operating across boundaries while providing enterprise-level security and business continuity.

VMware Hybrid Network Cross Cloud VMotion

Below is the two features of Project SkyScraper that was shared today at VMworld:

  • Cross-Cloud vMotion is a new technology based on vSphere vMotion that allows customers to seamlessly migrate running virtual machines between their on-premises environments and vCloud Air.
Read More

vRealize Automation Modifying Approval E-mails

One of the questions I get often asked by customers is how to modify vRA e-mail templates. To be honest the most one customers usually care to modify is the Approval e-mail to include more information than what is provided in the out of the box template. In this article, I will show you how to simply modify the vRA Approval e-mail to include additional custom properties. I will not drill into more advanced e-mail modification topics to keep this as simple as possible and get you straight to the point if that all you needed to do, though I will have more advanced vRA e-mail template modifications links at the end of this post.

First, you will need to upload the vRA e-mail templates to vRA using the below steps:

  1. Download the 2088805_vrealize_automation.tar.gz file from KB208805
    • Windows: Use a program such as WinSCP to copy the 2088805_vrealize_automation.tar.gz to the VMware vRealize Automation
    • Linux: Run these commands:scp 2088805_vrealize_automation.tar.gz root@vCAC VA:/
      ssh root@vCACVA
      cd /
      tar -xvzf 2088805_vrealize_automation.tar.gz
      find /vcac -type d -exec chmod o+rx {} \;
      find /vcac -type f -exec chmod o+r {} \;
  2. Restart VMware vRealize Automation by running this command:service vcac-server restart.

The above steps should get you to a position where you are ready to modify the vRA e-mail templates as needed, and changes to these templates going forward will take effect without needing to restart the service, but might take up to 2 minutes to see the changes.… Read More

vRealize Automation Gugent stuck and machine deployment timeout

After installing the vRA 6.2.2 Gugent into our Windows 2012 R2 template, blueprint deployment started to time out. As I had followed the exact steps I have previously used to install the Gugent for quite few customers before which in turn I have already documented on my following blog post: vRA Guest Agent Installation, I was surprised with the issue and thought to document the issue and resolution of it in this blog post in case others encounter it. Before I start, just few words of the environment versions:

vRA: 6.2.2
Gugent: 6.2.2
vCenter: 6.0
ESXi: 6.0
VMware Tools: 6.0
OS: Windows 2012 R2

While above these are the exact versions I have seen the issue with, you might face it with previous or future versions, as I am not sure yet which versions this specifically apply to, I wanted to ensure to point out the exact versions where I have observed it. Below is the timeout error I saw in vRA while trying to deploy from a blueprint that uses the Gugent: “Request failed: Machine vttest001: Timeout Customizing Guest OS.”

Request failed machine timeout customizing guest os

 

The error logs within the vRA portal was not providing me with enough info to debug the error with, so I started digging into one of the machines deployed through the problematic blueprint and after digging around I have found the following error in Windows Applications event log:

“faulting application name: DynamicOps.agent.guest.exe libeay32.dll”

 vRA Gugent Faulting Application name: DynamicOps.Agent.Guest.exe

For some reason the issue end up being caused by the vRA Gugent not being able to locate the VMware Tool libeay32.dll package, although VMware tool 6.0 was installed.… Read More

vRealize Automation deployment to vCloud Air fails

After configuring vRealize Automation Center to be ready to deploy to vCloud Air (End Point, Fabric Group, Reservation, Reservation Policy, Blueprints, etc), my initial vAPP blueprint deployment to vCloud Air has started to fail with the below error message.

“Request failed: Machine VTTest001: an error occurred while customizing the vapp..”

vRA vAPP deployment to vCloud Air fails error occurred while customizing the vAPP

 

Checking the vRA logs under monitoring has given me a bit more descriptive error, which has definitely helped me resolve the issue much more than the original error: “vCloud Customize VM failed for vApp: VTTest001 Error occurred while customizing vAPP. Inner Exception: Auto-logon count must be within 1 to 100 range if enabled or 0 otherwise.”

vAPP deployment fails in vRA autologin error under monitoring log

This has reminded me that I has never setup the auto-logon setting under customization on the VMs building my vAPP template. You will need to check mark the “Automatically log on as Administrator” option as shown in the below screenshot. I would leave the number of automated log on to 1 as well.

vRA deployment to vcloud Air fails Automatically login as an administratorThis has done the trick for me and things worked afterward. I assume others might miss this step as well, and as the initial error might not be obvious indication of the issue, I thought I will try to save you the hassle of trying to figure out how to fix it.… Read More

vRealize Automation iaas-service does not register after installation

While doing the usual vRA 6.2.2 large deployment distributed install, and after everything in the installation went smoothly without any errors, the vRA infrastructure tab was not working. Checking the vRA Services through the vRA Appliance VAMI, and noticed all of my services are up and running as  expected with its status being registered, beside the iaas-service which its status was left blank. Look at the below screenshot and notice where the service column ” iaas-service        com.vmware.csp.iaas.blueprint.service” status being blank.

vrealize automation iaas-service not starting

Going through every test I can think of, I could find no meaningful error in any of the logs, till I opened my infrastructure tab again but in Chrome this time and rather than getting the usual 404 error it was giving me earlier the below message came up: Invalid Argument. Host name may not be null

vra Infrastructure tab Invalid Argument Host name may not be null

 

After a bit of digging/research, I have found out that the VIP names the customer is using included under scores which is not supported by vRA for both hostnames or VIP names. This has been documented in the vRA 6.2 Installation and configuration document here as well in KB2086553.

Resolution for this issue: You will need to avoid this scenario before hand, else you will have to redo the full installation.… Read More

vRealize Automation 6.2 moving to Internal replicated vPostgres

When delivering a vRealize Automation 6.1 and prior in a distributed install, the recommendation was always to have an external vPostgres instance. This has changed in vRealize Automation 6.2 and it’s briefly touched upon in the vRA 6.2 reference architecture found at: vRealize Automation 6.2 reference Architecture. I wanted to make sure to highlight this, as many have missed this change and others are still wondering how it can be implemented or why this change. It’s important to mention as well, you can use this replication architecture for any version of vCAC/vRA that’s higher than 6.0.1.

Let’s start by highlighting the change, we are moving from having a separate/external vPostgres  machine to host the vPostgres (being a vRA appliance that is re-purposed to be used as vPostgres or a full vPostgres install) to using the internal built-in vRA Appliance vPostgres and replicate across the two vRA Appliances. The below image demonstrate this change:

vRA internal vPostgres replication
So what do you get of moving to this new model? Actually there is few advantages of implementing your vRA 6.2 this way, below is couple of them.

1- You have less machines to manage by eliminating the extra one or two external vPostgres machines you needed in the 6.1 reference architecture setup.… Read More

Sponsors