vRealize Automation Gugent stuck and machine deployment timeout

After installing the vRA 6.2.2 Gugent into our Windows 2012 R2 template, blueprint deployment started to time out. As I had followed the exact steps I have previously used to install the Gugent for quite few customers before which in turn I have already documented on my following blog post: vRA Guest Agent Installation, I was surprised with the issue and thought to document the issue and resolution of it in this blog post in case others encounter it. Before I start, just few words of the environment versions:

vRA: 6.2.2
Gugent: 6.2.2
vCenter: 6.0
ESXi: 6.0
VMware Tools: 6.0
OS: Windows 2012 R2

While above these are the exact versions I have seen the issue with, you might face it with previous or future versions, as I am not sure yet which versions this specifically apply to, I wanted to ensure to point out the exact versions where I have observed it. Below is the timeout error I saw in vRA while trying to deploy from a blueprint that uses the Gugent: “Request failed: Machine vttest001: Timeout Customizing Guest OS.”

Request failed machine timeout customizing guest os

 

The error logs within the vRA portal was not providing me with enough info to debug the error with, so I started digging into one of the machines deployed through the problematic blueprint and after digging around I have found the following error in Windows Applications event log:

“faulting application name: DynamicOps.agent.guest.exe libeay32.dll”

 vRA Gugent Faulting Application name: DynamicOps.Agent.Guest.exe

For some reason the issue end up being caused by the vRA Gugent not being able to locate the VMware Tool libeay32.dll package, although VMware tool 6.0 was installed. To work around this we end up copying the libeay32.dll package from the VMware tools folder to the vRA Gugent “VRMGuestAgent” folder as per the below screenshots.

VMware tools copy libeay32.dll into the Gugent folder

copied the libeay32.dll to VRMGuestAgent

This seemed to fix it for me, but I am still not sure why the vRA Gugent was not able to locate the VMware Tools libeay32.dll package, although my VMware tools was installed in the default folder. I will be further investigating this, but till then, this should get you going.

Please note, while only copying the libeay32.dll to the “VRMGuestAgent” folder seemed to do the job when using vSphere 6 and the template was running the vSphere 6 VMware tool, it seems there is one more file required when running with older VMware tools which is ssleay32.dll. I would recommend even in vSphere 6 you copy both files. You can get both of these files from the VMware tools folder if you are using vSphere 6.x or you can get them from an older version of the Gugent like the vRA 6.2.1 Gugent.

Comments

  1. Hi Eiad,

    Have seen this a couple of times now; with 6.0.1 and again with 6.2.2. Have also found the need to copy ssleay32.dll into the gugent directory to resolve. Another way of getting to see the error is to stop the vrmguestagent service and then run doagentsvc.bat from the gugent directory and wait about 30 seconds or a minute and you should get a popup error

    Hth
    DB

  2. Rishab Mehta says

    Hi,

    I am facing the same problem here, but i am not able to find the file you have listed in the VMware Tools folder.
    Here is my setup details
    ESXi 5.5
    vCenter 5.5
    vRA 6.2.2
    OS 2012

  3. Hi Rishab, I was using vSphere 6, and not sure if the same issue exist for vSphere 5.5

  4. Same issue here. vSphere 5.5, Guest Agent 5.2.2. VMware tools from vSphere 5.5 don’t include the DLL for some reason?

  5. Sorry, that should have read *6.2.2*

  6. Pardha Nallan says

    Hi Eiad,

    This worked for me. But one caveat here is that I was unable to find the files in the VMTools folder. However I had another vRA setup with 6.1 and I grabbed these files from there and copied them to the VRMGuestAgent folder and things started to work again.

    Alex – You may want to try this if you have another setup…

    Thanks a lot Eaid. Your blog certainly helps a bunch!!

    Pardha.

  7. Seeing this same issue in vRA 7.01. Hard to call this enterprise ready…

  8. Hi dd, Can you please confirm which version of OS are you running and what is the version of the VMware tools are you using? As I have not seen this issue since vRA 7.0 and I am wondering which versions it’s affecting if any in 7.0/7.0.1.

  9. We are running server 2012R2 w/ version 10 tools and OEL6 as our guest VMs. I was able to successfully implement the fix for Windows and have some 60 deployments w/o any Timeout Customizing Guest OS errors. However, I’m now trying to find the same type of fix for OEL6. I’m thinking of copying the vmware-tools directory into the guest agent to see if that works.

  10. Even, we have similar issues with our vRA7.0, vCenter Appliance 6.0, this happens for both windows and Linux VM’s.

  11. Even we have facing the same issues.
    vRA: 7.0
    vCenter: vCSA6.0
    ESXi: 6.0
    OS: Linux OEL 6u6 and Win2K12

  12. HI,

    I am seeing the same issue in vRA 7.2, and tried copying both the DLL but no luck.

    VM is stuck in CustomizeOS state.

    Customer is using vSphere 6

  13. Hi Muk,

    Make sure your VMs trust your root certificate as the issue addressed by this post does not really exist in vRA 7.2. Your issue is more than likely either due to certificate or connectivity issue.

    Regards,
    Eiad

  14. Hi Folks,

    I am facing the same issue “Timeout Customizing Guest OS” in the environment running vRA7.3 with vSphere 5.5. I am using one blueprint for provisioning in 2 different datacenters with valid templates. The strange behaviour what I noticed was that it successfully provisions in one DC but fails with the timeout in other DC. There are no firewalls between. Did anyone face such issue in 7.3 or any recommended solution?

    Thanks

Trackbacks

  1. […] Automation Gugent stuck and machine deployment timeout This is one I have had issues with.  In fact I had one like this in my vRA training class!  So good info to know and […]

Speak Your Mind

*