How to fix: vCAC 6 AD Login is very slow

While working at a customer site with a pretty decent size Active Directory where they have implemented an Empty root structure, vCAC login through Single Sign On when using Active Directory accounts were pretty slow. It took up to 10-15 minutes at time per login attempt.

The customer had a multitiered domain in a single forest. The top domain/root domain or if you want to call it level 1 domain was empty, where all the users and where vCAC users were coming from is the level 2 domain. What was happening is when a user of the level 2 domain try to login he was facing one of the below two problems:

1- If the user does not have any group membership outside  the level 2 domain, they were allowed to login, but it took quite a bit for the login attempt to complete. (10-15 minutes)

2- If the user has any group membership or any tie outside the level 2 domain, after the log on attempt, the progress bar on the log on page will stop and nothing will happen.

At first, I thought I might be short at resources on my SSO server so I boosted that up, while it speeded things a bit it was not too noticeable of improvement and I knew there was a bit more work to do.… Read More

What’s New in StarWind SAN V8

StarWind V8 RCStarWind SAN V8 RC has just been released with a lot of new features. Below is a summary of what to look for in this release:

Log-structured File System:

• VAAI UNMAP command implemented.
• Internal structures changed to ensure compatibility with upcoming updates.
• Improvements made for operations with VSS providers.


Flash cache:

• Optimizations and fixes for cache operation added.
• WB mode for L2 cache returned.


• Added VSS support for devices with synchronous replication.

NAS Configurator:

• NAS Configurator utility helps to export storage as SMB or NFS network share by means of StarWind High Availability and MS Cluster

VSA Builder:

• VSA Builder utility deploys StarWind VSA virtual machine on VMWare ESXi server.

V2V Converter:

• V2V Converter utility added to StarWind Management console. It is a V2V conversion tool for virtual machines.You can use it to convert VMDK to VHD files and VHD to VMDK as well as to IMG file, which is a native StarWind format.

Management console:
• Scale-out functionality extended: updates for device views and device tree.
• Fix for file naming: now wizard generates header file name and image file name for L2 cache storage according with device header file name.
• Fixed bug connected with the creation of new folder in file browser.… Read More

Warning: The current browser that you are using is not supported for use with vCAC 6.0

After setting up vCloud Automation Center 6.0.1 for one of my clients, I was able to access the vCAC portal using my Firefox browser with no problem what so ever. After I have finished the configuration of vCAC and turned the setup to the customer to try it from his machine, he did not have Firefox and their corporate policy force them to use IE8 which is by the way a supported vCAC browser and I was confident it should work there with no problem. The surprise was as soon he opened up the vCAC page in his IE8, he got the following error:

‘Warning: The current browser that you are using is not supported for use with vCAC 6.0′

vCAC 6 not supported error

This has got me in doubt if I have conveyed the wrong message to the customer of IE8 being a supported browser, so I went right a head and checked vCAC 6.0.1 supported browsers list which can be found on page 14 at:

The list of supported vCloud Automation Centers as extracted from the document is copied below:



Browser Support

  •  IE8
  •  IE9 **
  • IE10 **


  • Chrome 29
  • Chrome 30


  • Firefox 23
  • Firefox 24

** Compatibility mode for IE9 and IE10 is not supported.… Read More

Win Free VMworld Trip with Veeam

I am sharing this news, as I am sure many of you are looking for a free way to get to VMworld, and Veeam is offering a few of you a chance to do that. They are even offering a chance to Win a Google Glass. Sorry Apple, but I would rather the Google Glass on an iPad my self! Below is the details on why Veeam is doing it and on how to register. Good luck!

Veeam is about to get its 100,000th customer and is launching an interactive contest for a chance to win a trip around the world and other prizes (Google Glass, iPad and Microsoft Surface).

To participate, visitors need to register and predict the location of Veeam’s 100,000th customer on the interactive map. The closer you are to the right spot, the better chance you have to win the trip around the world and other prizes.

Veeam currently have a live pre-registration page. The main contest will start next week.

Join in on the fun! I hope many of you will participate and I hope among my readers there will be few winners.

Just for the fun of it, if you have got a chance to choose between any of these gifts, which one will you choose?… Read More

Vote For Your Favorite Virtualization Blog

It is that time of the year again where you can vote for your favorite blogs at: It is a great way to show your appreciation to your favorite blogs and bloggers who you benefited from their blog posting. Remember most bloggers including myself are doing this on their spare time, and don’t get much of compensation in return so the least you can do is show your appreciation and vote for your favorite Virtualization Bloggers. For those of you who already know what they should do, then please head on to the following URL and vote for my blog:

Vote for your favorite blog

If you have enjoyed my blog posts for the past year, or found something that saved you from trouble then please don’t forget to vote for me. Your vote means a lot to me, and will definitely encourage me to drive my blog even stronger as to grows. I believe the competition will be quite tough this year especially with blogs like the following on the list:

  • (Hany Micheal)
  • Long White Virtual Clouds (M. Webster)
  • Yell0w Bricks (Duncan Epping)
  • ESX Virtualization (Vladan Seget)
  • Virtually Ghetto (William Lam)
  • Kendrick Coleman (Kendrick Coleman)

Actually these are few of my favorite Virtualization blogs, and though it might give you an extra choice or two to vote for if you ran out of option during your choice of top 10 blogs.… Read More

vCenter Appliance fsck failed

While I have been working with a customer on upgrading their infrastructure to vSphere 5.5, they had a power outage during the night which got their full virtual infrastructure down including storage (Not sure what the story with their UPS/Power generator). After they have brought their virtual infrastructure backup up both their vCenter Appliance and VDP appliance and few other Linux VMs has failed the fsck check while it was booting and it was stuck at the following error:


fsck failed.  Please repair manually and reboot.  The root
file system is currently mounted read-only.  To remount it
read-write do:

bash# mount -n -o remount,rw /

Atention:  Only CONTROL-D will reboot the system in this
maintanance mode.  shutdow or reboot will not work.

Give root password for maintenance
(or type Control-D to continue):


Below is an actual screenshot of the error for you to compare it with what you got:

VMware vCSA fsck failed

Below is the steps I have followed to fix this on both vCSA and VDP Advanced:

  1. Stay Calm & Don’t Panic!
  2. Take a backup/snapshot in case the below steps does not work for you and you need to revert it back to the current state when you call support or call for help.
  3. Read More

vSphere Data Protection Advanced Licensing your hosts

I have seen a consistent growth lately in the number of my customers that shift to using vSphere Data Protection (VDP). I guess with VDP standard edition being a freebie nonetheless all the powerful features it come up with has made it an easy choice for many customers in the SMB market. VDP standard comes loaded with many features including: (Find out more about each of these features at: )

  • Variable-length de-duplication
  • Global de-duplication
  • Changed Block Tracking backup (CBT)
  • Changed Block Tracking restore
  • One-step recovery

Though as with every free thing in life it comes with few limitations, where most of them are waived when upgrading to the paid version called VDP Advanced. Below is the main limits of the free VDP Standard Edition:

  • Each VDP appliance supports backup for up to 100 virtual machines..
  • Support a maximum of 2 TB of deduplicated backup data.
  • Each vCenter Server can support up to 10 VDP appliances.

Upgrading to the vSphere Data Protection Advanced Edition waive most of these limits as below:

  • Each VDP Advanced supports backup for up to 4,000 VMs in comparison to only 100 in the Standard Edition
  • vSphere Data Protection Advanced expand the maximum supported deduplicated backup datastore to 8TB from 2TB in the standard edition
  • vSphere Data Protection Advanced still limited to 10 VDP appliances per vCenter.
  • Read More

vCenter Single Sign On 5.5 Whats New

vCenter Single Sign On has a considerable amount of changes in vSphere 5.5, with few major ones. Many of these changes have went undetected or unnoticed by the Virtual Infrastructure Admins. If you have deployed vSphere 5.5 and missed these changes or planning to install vCenter SSO 5.5 and want to learn what has changed from the vSphere 5.1 days, then this post is for you:

Below is the list of the major changes introduced in vCenter Single Sign On:

 vCenter SSO Architecture Improvements:

1- Multi master: Unlike 5.1, vCenter SSO 5.5 has A built-in automatic multi-master replication architecture that ensure that all SSO instances are always kept in sync. While this sound great, most admins are not sure what it means nor how it affect the way they design SSO. To understand the value of this change, you must understand how SSO worked in 5.1. and how that changed in 5.5.

In 5.1 if you wanted to enable SSO for multiple vCenters in your environment, you needed to point all of them to the same SSO instance which made all of those vCenters dependent on that single SSO instance. This has made that single SSO instance to be crucial for the operation of all of your vCenters, as if that SSO instance goes down you will not be able to access any of your vCenters.… Read More

vCenter Server Appliance Error: failed to connect to VMware Lookup Service

While working with VMware vCenter Server Appliance 5.5, I needed to change the IP Address of the appliance as my customer were changing their IP scheme. It was quite to go to the VMware vCSA then to the network tab then Address tab and change the IP Address. After rebooting the appliance to ensure the IP change has taken affect and is working properly and consistent, I was able to login to the vCenter Server Appliance Management portal, but I was not able to connect to using the Web Client and was kept getting the following error:

“Failed to connect to VMware Lookup Service https://<vCenter Appliance IP>/lookupservice/sdk – SSL certificate verification failed”

Or as shown in the below screenshot:

Web Client Error after changing vCenter Appliance IP


It seems changing the IP Address of the vCenter Server Appliance does not regenerate the certificate with the new IP and you will have to configure the appliance to generate a new one manually. The good news it is pretty easy to do, and below is the instruction:

  1. Log in to the vCenter Server Appliance Web interface at https://<vCSA IP>:5480/.
  2. Click the Admin tab.
  3. Regenerate certificates:
    • vCenter Server 5.1: Select Toggle certificate setting so that the Certificate regeneration enabled displays Yes.
  4. Read More

vSphere Data Protection error – operation failed due to existing snapshot

After I have helped one of my customers deploy VMware vSphere Data Protection Advanced to replace their old VMware Data Recovery, and helped them schedule their environment for backup, we came back the next day to find out that few VMs have failed with the following error: “VDP: operation failed due to existing snapshot”.


As most of our VMs backed up successfully we were pretty sure it is not the VDP software, and that the problem is related to these specific VMs. It end up just like the error is indicating these VMs have Snapshots that have not properly been removed and needed to be cleaned up before VDP can backup these VMs. Below is the action we have taken to fix this problem:

  1. Check the Snapshot Manager for VMs that failed to backup.
  2. If Snapshot Manager showing any snapshots delete them, then proceed to step 4.
  3. If Snapshot Manager not showing any snapshots, just proceed to step 4.
  4. After deleting all snapshots showing in snap manager, create a new snapshot.
  5. Hit Delete All in Snapshot Manager after the new snapshot have been completed.

The above procedure in most cases should clean up your VM from none consistent dirty snapshots, and you should be able to backup these VMs after that using VMware VDP.… Read More

vCenter Server Appliance 5.5 limitations

VMware vCenter Server Appliance 5.5 has been gaining huge attraction lately, especially with many of the major limitations affecting the vCenter Appliance in previous verions have been lifted. This is including the limitation where it was only supposed to support 5 hosts and 50 VMs when using the embedded database. In fact the all shiny new VMware vCSA 5.5 support up to 100 hosts and 3000 VMs using the embedded database, which seems to be able to deliver up to the scale required by most customers.

Note: It seems a misleading assumption has been going around that 100 hosts and 3,000 VMs is the maximum supported configuration by VMware vCSA 5.5, where is that is absolutely not true. This limit only apply when using the embedded  built-in vPostgres where it actually can support up to the vCenter maximums of 1,000 hosts and 10,000 VMs when connected to an external Oracle Database. This is clearly documented on page 7 of vSphere 5.5 Maximums guide and a copy of that table is shown below. I believe this misleading belief has spread as many bloggers has mentioned the 100 hosts and 3,000 VMs limit while failing to point out it is not the actual limit of the appliance, but it only apply when using the embedded database. 

VMware vCenter Server Appliance Maximum Configurations

I have actually been using VMware vCenter Server Appliance 5.5 for most of my labs and at various customers as it deliver what they needs, as it can be deployed in fraction of the time and save them from having to build Windows and DB servers, not to mention its easier to maintain and manage.… Read More

Deploying VMware vCenter 5.5 Appliance

Lately for most of my labs and even for many customers I have been deploying the VMware vCenter 5.5 Appliance, as it is much quicker to install and give less chances for error. Further it saves customers a Windows license and not to mention being much easier to maintain, patch and upgrade. Further, with many of the earlier versions limitations being lifted it becoming much more viable option than ever before. For that I have decided to document the process of installing the VMware vCenter 5.5 Appliance in a hope to encourage more of you to try it out.

 VMware vCenter 5.5 Appliance Installation Steps:

1- Download the VMware vCenter 5.5 Appliance OVA (I rather the OVA as all you have to get and deploy is a single file)

2- Deploy the VMware vCenter 5.5 OVA into one of your hosts.

  1. Launch the vSphere Client or vSphere Web Client.
  2. Go to File > Deploy OVF Template.
  3. Browse to the location of the vCenter Server Appliance downloaded .ova or the .ovf file and click Open.
  4. In the OVF Template Details page, click Next.
  5. In Name and Location, enter the name of your vCenter Server Appliance and click Next.
  6. Read More

VMware vCloud Hybrid Service vSphere Client Plug-in is here

VMware has just announced vCloud Hybrid Service vSphere Client Plug-in which will help VMware customers manage their VMs in VMware Hybrid Service the same way they manage their vSphere environment right from inside their vSphere Client (Only Web Client as you would guessed), which should increase administrators efficiency and reduce the time of learning a new interface as they will get to use the vSphere familiar interface to manage their VMs in the Cloud. vSphere Client will be your single pane of glass to VMs running in your datacenter and in VMware Hybrid Service. This is such a great achievement and will complement VMware Hybrid Cloud story.

vCHS vSphere Client Plug-in

vCloud Hybrid Service vSphere Client plug-in extends the familiar vSphere Web Client UI to vCloud Hybrid Service. With the vCHS plugin, from within the vSphere Client, customers can:

  • View and administer Dedicated Cloud and Virtual Private Cloud instances on vCHS
  • Browse, create and modify each cloud instance and manage its inventory of virtual data centers, gateways, and networks
  • Deploy virtual machines using catalog templates and modify existing VMs
  • Receive automatic updates on any changes made to their account via vCHS web portal



  • vSphere Web Client 5.5  (No C+ Client)
  • vCloud Hybrid Service account with VPC or Dedicated cloud
  • Compatible Browsers:
    •   Windows: Firefox, Chrome and Internet Explorer
    • Mac: Chrome and Safari   (VMs Console Access does not work on Mac)
    • Linux: Firefox and Chrome


Enjoy the new enhanced Experience of VMware vCloud Hybrid Service!

Read More

Qtree SnapMirror warnings and limitations with SRM

While setting up SRM with NetApp 6290 at a customer site, my customer was using Qtrees and Qtree SnapMirror which caused us few issues. If you are setting up SRM and using NetApp Qtree SnapMirror, there is quite few warnings, limitations, and best practices that you will need to be aware of. I have listed the most common ones below, though for a more complete list you should check the following document:

- Avoid using hidden Qtrees as that seems to cause problems with several versions of the NetApp SRM SRA. One of the most common errors caused by such configuration is:

Error: Failed to sync data on replica device ‘/vol/volume_name/lun#. Device found is neither of SAN type nor of the NAS type. Ensure that the device exists on the storage array and is of type NAS or SAN. 

RM NetApp Hidden Qtree error

- If you have configured qtrees as NFS datastores, you must create an NFS export for each qtree in order for SRM to be able to discover the NFS datastore. If you export only the volume that contains the qtrees, so that there is only one export line for the volume in the /etc/exports file, SRM will not be  able to discover the qtree NFS datastores.… Read More

SRM Error: XmlValidateException Element SourceDevice is not vaild for content model: Source Device.

While setting up vCenter Site Recovery Manager 5.0.3 with NetAPP 6290 filer,  & in particular at the stage where I try to enable the array pair and after I fixed the timeout error I have documented in the following post: SRM Time Out (300 seconds) while waiting for SRA to complete discoverDevice Command, I have faced this error:

Internal error: std::exception ‘class Dr::Xml::XmlValidateException’ “Element ‘SourceDevices’ is not valid for content model: ‘(SourceDevice,)”‘.

A screen capture of the error is below:


SRM NetApp 6290 element SourceDevice is not valid for content model

After a bit of digging I have found out that is a well known NetApp SRA 2.0.1 bug which is documentd here, and there is few possible workaround. Below I will document these workarounds, where you can take the most feasible option for your environment.

1- Revert back to NetApp SRA 2.0.0 as that does not suffer the same problem. This bug only seems to affect NetApp SRA 2.0.1 not 2.0.0. This is actually the solution I went with as it required the least amount of work and maintenance going forward.

2- Download and install NetApp SRA Patch 2.0.1P2. This patch does not seems to be generally available and you will have to request it from NetApp support or digg for it into the NetApp support portal.… Read More