vCloud Automation Center 6.x Time Bomb Issues

As I know quite few of my vCloud Automation Center 6.x (lately renamed to vRealize Automation) customers are following my blog, I wanted to share this important information of a two different time bombs issues that affect the 6.x releases of the product.  The purpose of this post is to highlight these two time bomb issues and the simple changes required to avoid them affecting the availability & manageability of your vCAC environment. If you have not done this or not sure if it was done during the installation time, please take the time to do it now. It is worth checking even if your environment was setup by a vCAC expert as it might have not been known during the the time your setup was installed:

1- vCAC 6.x VAMI Root account time bomb.

In vCAC 6.x There is a one year expiration time bomb in the VAMI Root account. This will prevent you from logging in to the VAMI consoles. To fix run this command…

# chage –M 99999 root

 This will change the timeout  and should prevent future issues. This occurs in both the vCAC Identity Appliance , and the vCAC Appliance.

2- vCAC 6.x SSO internal tenant admin password  time bomb.Read More

VMware vRealize Automation 6.2 is now available for download

vRealize Automation 6.2, which is the new name for vCloud Automation Center has been made available for download earlier today. This release will be adding a lot of new enhancement & features where you can find the full list in the vRealize Automation 6.2 Release notes. In here I want to cover few main enhancements, that I was patiently waiting for.

1- Ability to edit custom properties for published applications in the service catalog. While this enhancement is sneaked in very late down in the release note & many have not given it as much attention, it is one of the greatest enhancement in my opinion in this release. If you have worked with vCAC in the past and tried to integrate it with VMware Application Director (just renamed to vRealize Automation Application Services), you were always faced with the challenge that if you publish applications using application director into vCAC, you will not be able to pass custom properties to the underlying VMs blueprint, which was a big challenge. Actually in many cases, it eliminated the ability to use application director without a very heavy customization. Now this constrain being lifted, I can see the amount of great power vRealize Automation will enjoy when coupled with vRealize Automation Application Services.… Read More

vCAC Guest Agent Installation

While using VMware vCloud Automation Center Guest Agent or as many call it Gugent is not a mandatory requirements to use vCAC, it has its merits and use cases. In fact, many customers don’t end up deploying the vCAC Guest Agent within their Windows & Linux machines from day one, as not having the agent won’t stop vCAC from deploying a Windows or Linux VM and carry out the standard OS Customization. The interesting fact though, most of them end up installing it within their templates couple of months later, as they come up with a use case or two for it.

Let’s start by going over the vCAC Guest Agent use cases, and why should you install the Gugent within the templates you want to deploy with vCAC:

vCloud Automation Center Guest Agent Use Cases:

Launch in guest Scripts, this is has always been the biggest driver for installing the vCAC Guest Agent. The nice part about it, after installing the guest agent, you can customize which script you want to run and what parameters you want to pass to it using custom properties. Further being able to invoke these scripts from share drives and passing them your desired parameters using custom properties,  you can avoid having to edit your template, every time you update your script.… Read More

How to use vCAC new NetworkProfileName Custom Properties

Before the release of vCloud Automation Center 6.1.1, it was common to combine the use of the two below custom properties to assign a particular virtual machine to a particular portgroup/Network Path and a particular network profile:

VirtualMachine.NetworkN.Name: This custom property is used to put the virtual machine network adapter N, on the portgroup name supplied as a value for this custom property.

VirtualMachine.NetworkN.ProfileName: This custom property is used to tell the virtual machine network adapter N to obtain an IP from the network profile named in the value of this custom property.

I have seen the combined use of these two custom properties many time in the past and they seemed to work properly before vCAC 6.1.0 (It might have stopped working a bit earlier than that but I did not notice it). On the other hand using each of these custom properties on its own still work properly in vCloud Automation Center 6.1 and beyond, combining both custom properties on the same blueprint seems to produce some odd behaviors and unexpected results after 6.1. To avoid having such a problem, its highly recommended to use the newly introduced VirtualMachine.NetworkN.NetworkProfileName custom property.

vCAC NetworkProfileName custom property

VirtualMachine.NetworkN.NetworkProfileName kinda combine both custom properties in a single property.… Read More

Windows could not parse or process the unattend answer file for pass [specialize]

I was at a customer which I have deployed vCloud Automation Center 6.1 for, and they tried to create a new blueprint. They have copied the old blueprint and only changed the VM template used in that blueprint from Windows 2012 to Windows 2008 R2. They saved and published the new blueprint without any problem, then they tried to deploy the new blueprint. The blueprint deployment kept failing with the following error:

“Request failed: Machine VTTestVM001: Timeout customizing machine”

From the error, I was able to tell it was getting stuck at the guest customization stage, so I deployed one more VM and waited till it get to the guest customization stage, then opened a console to the VM to be greeted  with the following Sysprep error:

Windows could not parse or process the unattend answer file for pass [specialize]. The settings specified in the answer file cannot be applied. The error was detected while processing settings for component [Microsoft-Windows-Shell-Setup]“

Below is a screenshot showing the actual error on screen:

Windows could not parse or process the unattend answer file or pass [specialize]

After I hit OK, the machine restarted again and gave the following error and kept repetitively restarting and giving the same error afterword:

“The computer restarted unexpectedly or encountered an unexpected error.Read More

vCAC 6 Property Dictionary Relationship Builder

While you can create property Dictionary Relationship XML by hand as I have shown in my previous post: vCAC 6 Property Dictionary Relationship, it has been proven to be so tedious of task to do when having a large environment,  that I wanted a more compelling way of doing it. That’s why I came up with my vCAC 6.x Property Dictionary Relationship Builder (This work with all versions of vCAC 5.1 and higher). I am going to continue with the example of my previous two posts: vCAC 6 Custom Properties – Build Profiles – Property DictionaryvCAC 6 Property Dictionary Relationship and show you how to generate the same property dictionary relationship XML using my my Property Dictionary Relationship Builder. Please read my previous post for more background of the scenario used in this example.

Let’s start by quickly showing the same table used in my previous post to demonstrate the relationship we are trying to generate. In this post I will use my generator to produce the location and storage relationship demonstrated in the below table

Location Storage Network
Toronto Toronto_T1 Toronto_Prod_PG01
Toronto_T2 Toronto_Prod_PG02
Toronto_T3 Toronto_DMZ_PG01
Boston Boston_T1 Boston_Prod_PG01
Boston_T2 Boston_Prod_PG02
Boston_T3 Boston_DMZ_PG01
Paris Paris_T1 Paris_Prod_PG01
Paris_T2 Paris_Prod_PG02
Paris_T3 Paris_DMZ_PG01

The expected Property Dictionary relationship XML file between Location & Storage for the above table look like below:

<?xml version=”1.0″ encoding=”utf-8″ standalone=”yes”?><ArrayOfPropertyValue xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Toronto</FilterValue><Value>Toronto_T1</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Toronto</FilterValue><Value>Toronto_T2</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Toronto</FilterValue><Value>Toronto_T3</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Boston</FilterValue><Value>Boston_T1</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Boston</FilterValue><Value>Boston_T2</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Boston</FilterValue><Value>Boston_T3</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Paris</FilterValue><Value>Paris_T1</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Paris</FilterValue><Value>Paris_T2</Value></PropertyValue><PropertyValue><FilterName>Vrm.DataCenter.Location</FilterName><FilterValue>Paris</FilterValue><Value>Paris_T3</Value></PropertyValue></ArrayOfPropertyValue>

While the above XML file look not too bad and could be edited manually, imagine if you have to generate the same type of XML for a large Property Dictionary Relationship that include 10s of filter values and for each filter 10s of values.… Read More

vCAC 6 Property Dictionary Relationship

Property Dictionary Relationship in vCAC 6 allow you to build a relationship between the different drop down menu boxes provided to your End User for input. This provide a more elegant and flexible solution when it come to user input in the vCAC request form. It will help you provide the user with only the valid choices rather than provide them with every choice in the list. Let’s take an example.

vCAC Property Dictionary Relationship

The above diagram demonstrate a vCAC request form with three drop down menus into it for Location, Storage, & Network. Let’s assume you have the following list of items for each list as below:

Location Storage Network
Toronto Toronto_T1 Toronto_Prod_PG01
Toronto_T2 Toronto_Prod_PG02
Toronto_T3 Toronto_DMZ_PG01
Boston Boston_T1 Boston_Prod_PG01
Boston_T2 Boston_Prod_PG02
Boston_T3 Boston_DMZ_PG01
Paris Paris_T1 Paris_Prod_PG01
Paris_T2 Paris_Prod_PG02
Paris_T3 Paris_DMZ_PG01

If you implement the above table values into a property dictionary drop down menus(Without building relationship), you will have 3 options to choose from for location, and 12 options to choose from for storage and network no matter what location you have chosen, although only 3 options for Network or storage is valid for any particular location. In this case, property dictionary relationship become a very handy feature. As you can filter the storage and network field based on the location field.… Read More

vCAC 6 Custom Properties – Build Profiles – Property Dictionary

vCloud Automation Center offer a lot of extensibility features built in within the product that help you achieve your desired result while minimizing the amount of coding required. vCAC Custom Properties, Build Profiles, Property Dictionary is just an example of how you can customize the product to meet your needs while minimizing coding as well customize the input form to meet your need. As Property Dictionary seems to be the most missed/mis-understood feature of vCAC followed by Build Profiles and Custom Properties, I have decided to put a post together trying to simplify the explanation of these great features as much as possible. As well, I will be pointing you out to more resources at the end of the article to learn more in depth of each of these features.

vCAC Custom Properties:

As Custom Properties is the building block for Build Profiles and Property Dictionary, it will be very hard to explain any of the later without covering Custom Properties. As per VMware documentation, the way they define custom properties is as follow:

“VMware vCloud Automation Center™ custom properties allow you to add attributes of the machines your site provisions, or to override their standard attributes.”

What that is trying to say, vCloud Automation Center utilize particular variables (Custom Properties) that contain values that vCAC uses during machine provisioning (Ex: machine name, machine IP Address, Port Group to use, & so on).… Read More

vCloud Automation Center 6.1 Upgrade (Step-by-Step)

As the long waiting vCAC 6.1 was released beginning of this week, I have decided to test the upgrade process in my home lab as many of my customers are getting ready to benefit of the new features and enahancements in this release. The good news for everyone that the process is straight forward, where the only challenge I had during the upgrade is satisfying the Java requirements. This post will take you in a step by step fashion of the vCAC 6.0.1 to vCAC 6.1 upgrade process.

Upgrading the vCAC Appliance

  1. Shutdown the services on the vCAC Appliance using the following commands:
    • service vco-server stop
    • chkconfig vco-server off
    • service vcac-server stop
  2. Stop the vCAC Appliance before upgrade
  3. Shutdown all the services on your IaaS Windows Server. Under Start => Administrative tools => Services. Make sure to shutdown:
    • All VMware vCloud Automation Center agents
    • All VMware DEM workers
    • VMware DEM orchestrator
    • VMware vCloud Automation Center Manager Service
  4. You can download the update by going to the vCAC Appliance and let it directly download the vCAC update from the internet (What I am going to do here). Else you can actually download an iso image or create your own web repo that you connect the appliance to for update.
Read More

Convert your vCAC Appliance to a vPostgres DB Appliance for vCAC Distributed install in 5 easy steps

VMware vCloud Automation Center (vCAC) 6.0 documentation and reference architecture recommends the utilization of an external instance of vPostgres when setting up a large/Medium distributed vCAC environment. However, since the release of vCAC 6.0 documentation, standalone vPostgres is EOA and no longer available as a standalone product. VMware has addressed that by allowing you to convert a vCAC 6.0 Appliance into a vPostgres Appliance for free. In fact, VMware has released a KB to demonstrate this, but that KB is highly focused on how to get replication setup for the vPostgres Appliance rather than on how to convert or use the vCAC Appliance in place of a vPostgres Appliance. KB2083563 is the article that show you how to use the vCAC Appliance in place of the vPostgres Appliance and how to setup replication for it. If you just need the simple steps of how to use the vCAC Appliance in place of the vPostgres Appliance and VMware HA is enough for your availability and you don’t require replication, then today is your lucky day as this article is going to show you how to achieve that in 3 easy steps (from now on in this article I will call the vCAC Appliance to be used as a vPostgres database vCAC vPostgres appliance):

  1.  Download vCAC 6 Appliance & Deploy it to be used as your vCAC vPostgres Appliance.
Read More

How to configure vCAC not to delete VMs after deployment failure

I have a customer approach me asking of how we can stop vCAC from deleting VMs after deployment has failed for debugging purposes. The default behavior of vCAC is to delete the VM after deployment failure even if that failure was just caused by Guest customization failure or by the Guest Agent execution failure. The problem when a deployment fail due to a Guest Customization failure or by Guest Agent execution failure and the VM get delete as the deployment fail, you are left with no in guest logs to trace the cause of the problem which explain the need for stopping vCAC from deleting VMs after VMs deployment failure for debugging/troubleshooting.

Luckily vCAC Proxy Agent can be configured not to delete VMs after deployment failure and send it to VRMDeleted folder instead of deleting it and it can be deleted manually after you finish your debugging. If you think you need to stop vCAC from deleting VMs from vCenter after deployment failure for troubleshooting or any other reason, then below is how to do just that.

A system administrator can modify proxy agent configuration settings, such as provisioning machine credentials and deletion policy for virtualization platforms, after installation. The proxy agent utility can be used to modify the initial configurations that are encrypted in the agent configuration file.… Read More

vCAC 6 Service Temporarily Unavailable

As I have deployed vCloud Automation Center 6.0.1 in my home lab on vSphere 5.5, the installation worked flawlessly as I have carried out the same installation several times in production for several customers before. vCAC seemed to work without any problem at all for few hours, then I started getting the following error every time I tried to access my vCAC Portal:

———————

Service Temporarily Unavailable 

The Server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

———————

Below is a screenshot of the actual error:

vCAC Service temporary unavailable

 

I was surprised to see the above error, and when checked the status of my vCAC Services under the services tab in the appliance management console of vCAC the pages was not loading although all the other tab were working just fine. I logged in through ssh to the vCAC Appliance and restarted the vCAC Service using the following command:

# service vcac-server restart

After restarting the vCAC Service and waiting for about 5 minutes, I have noticed that all the services of vCAC is now showing up on the service tab and right after that the vCAC portal started to function again. Initially I thought the service might hanged for some reason or so on, but after few hours the same issues came up, and I have noticed that I am getting the same problem repeatedly every few hours where my services stop again.… Read More

vCloud Automation Center and vCenter Appliance

Yesterday, I was asked twice if using our vCenter Server Appliance (vCSA) with vCloud Automation Center (vCAC) is a viable option. As one of the two requests came from inside our VCDX family, It seems to be not a common knowledge, not when I get asked by another VCDX about it.  I thought this seems to be a good topic to address in a blog post, as the following questions seems to come often:

  • Does vCloud Automation Center (vCAC) 6 support vCenter Server Appliance (vCSA)?
  • Do your recommend using vCSA with vCAC?
  • What’s the added benefits of using vCSA  in my vCloud setup?
  • What extra limitations does using the vCenter Server Appliance with vCloud Automation Center impose to my environment?

I am sure you either had one of these questions on your mind or had been asked one of these at least once if you are working on a vCloud Automation Center setup. I will try to address these questions in this post.

Does vCloud Automation Center (vCAC) 6 support vCenter Server Appliance (vCSA)?

vCloud Automation Center and vCenter Appliance

I guess the image above gave the answer away, but for those of you who liked it spelled out, yes vCAC 6 support vCSA. Just make sure you check the vCenter interoperability matrix to ensure you are getting a supported version of the vCenter Server Appliance with the version of vCAC you are planning to install.… Read More

vCAC 6 Missing VM Actions

This problems seems to lately surface more than not when delivery a vCAC 6 Distributed installation. After you complete your vCloud Automation Center 6 installation without a hitch and you thought everything is working perfectly, you go to your Catalog Entitlements and try to assign Actions allowed and you notice that you have only few of what you really should have. The below screenshot provide a sample of which only actions I were able to entitle to my catalog items:

vCAC 6 missing VMs Actions

 

Before you jump to a conclusion that this the same problem you are having, please ensure that you have assigned the right permissions to your blueprint and entitlements to your Catalog item. For initial testing try to give all permissions to your blueprint as in the below screenshot:

 vCAC provide permissions on Blueprints

After providing the permissions on the vCAC Blueprint then you want to ensure your Catalog entitlements are set correctly. Go to Catalog action entitlements and check which Actions could you entitle to your Catalog item. If it look like the below screenshot then you are definitely having the same problem this article is trying to address as you should have much more actions that you can entitle to your Catalog Items.… Read More

vCAC 6 Service Unreachable – Reference error REPO404

While delivering a distributed install of  vCloud Automation Center using exactly the same steps I have used for few previous engagements, & while the setup completed perfectly without any errors, accessing the infrastructure tab in vCAC has continuously reported the following error:

—————————————————————————-

Service Unreachable

A required service cannot be reached at the expected address.

Please contact your system Administrator for Assistance.

Reference error REPO404.

—————————————————————————–

vCloud Automation Center 6 repo404 error

I was quite certain I have done the certs right, as I followed my certs guide that I had followed in few other engagements previously and posted it on my blog before at: vCloud Automation Center 6 Certificates A to Z.  Just for your reference if you have not read that post all the certs was generated by Active Directory CA.

As the above error can be caused by few different causes, I have went into checking my different vCAC logs and the error that helped me identify the cause was in the vCAC IaaS Web Server Windows Event log and below is a copy of that error (I marked in red the part that gave it away)

——————————— vCAC 6 SSL/TLS secure channel Error Start ——————

Log Name:      Application

Source:        VMware GUI Administration

Date:          05/28/2014 5:36:52 PM

Event ID:      0

Task Category: None

Level:         Error

Keywords:      Classic

User:          N/A

Computer:      IAASWEB1.vt.com

Description:

Timestamp: 5/28/2014 9:36:52 PM

 

Message: Error occurred writing to the respository tracking log

System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.Read More

Sponsors