VMware vCloud Availability 3.0 Update

The long waited VMware vCloud Availability 3.0 will be released soon, so I thought to share some update about the exciting set of features of what’s coming. If you have been following up the evolution of VMware vCloud Availability, you will find many of your wishes and requests are materializing in vCAv 3.0.

VMware vCloud Availability 3.0 solution introduces a new and simplified product architecture for replication, disaster recovery and migration capabilities into a single product. With the service you can perform:

  • Migration and Disaster Recovery of VMs from on-premise vCenter Server to a vCloud site.
  • Migration of vApps and VMs between two Virtual Data Centers that belong to a single vCloud Director Organization.
  • Replication and recovery of vApps and VMs between vCloud Director sites.

vCloud Availability 3.0 Combined Solution

VMware vCloud Availability (vCAv) 3.0 is combining the features of originally three different products, as shown in the below table. It is replacing all of vCloud Director Extender, vCloud Availability for DR to Cloud, and vCloud Availability for Cloud to Cloud.

vCloud Availability 3.0 Combined Features


vCloud Availability 3.0 capabilities:

  • Management and monitoring of replications from on-premise vCenter Server to vCloud Director site.
  • Capability to migrate and failover workloads from on-premise vCenter Server to vCloud Director site.

vCloud Availability 3.0 On-Prem to Cloud

  • Capability to failback recovered workloads from vCloud Director site to on-premise vCenter Server site.
Read More

vCloud Availability for Cloud-to-Cloud DR 1.5 is here!

The vCloud Availability for Cloud-to-Cloud DR 1.5 has been made available today! 

vCAv C2C 1.5 has come with a number of great enhancements highlighted below:

vCloud Availability C2C 1.5

  • An improved user interface that can be embedded into vCloud Director and allows tenant and service provider users to perform DR operations and monitor DR environments from within vCloud Director.
  • Service providers can enable or disable incoming and outgoing replications for one or multiple vCloud Director organizations in source and destination sites.
  • Policies that allow service provider administrators to limit the following system attributes: a) the number of replications at the vCloud Director organization level, b) the minimum recovery point objective, c) the number of retained snapshots per VM replication, and d) the total number of VM replications.
  • A public API that enables the automation of user session management, vApp replication management and monitoring, as well as system monitoring.
  • A storage consumption report and an enhanced usage metering report.
  • Increased scale and performance.
  • vRealize Orchestrator Integration
  • vRealize Operations Day 2 Monitoring Pack
  • Integration with the VMware Analytics Cloud for the collection of telemetry via the CEIP program.
  • Added support for vCloud Director 9.5

For more information, you can find the release notes at: https://docs.vmware.com/en/VMware-vCloud-Availability-for-Cloud-to-Cloud-DR/1.5/rn/VMware-vCloud-Availability-for-Cloud-to-Cloud-DR-15-Release-Notes.html

Product Documentation: https://docs.vmware.com/en/VMware-vCloud-Availability-for-Cloud-to-Cloud-DR/index.htmlRead More

vCloud Availability Cloud to Cloud DR capabilities is here!

Today, VMware has announced the long waited  vCloud Availability for Cloud-to-Cloud DR 1.0. This is a thrilling news to many of our VCPP service providers partners who wanted a VMware native Cloud to Cloud DR/migration solution for vCloud Director based clouds.

vCloud Availability C2C

Below is few highlights of the newly introduced  vCloud Availability for Cloud-to-Cloud DR.

Simple Deployment:  vCloud Availability Cloud to Cloud DR is delivered as a single Photon-based appliance with a full GUI driven installation. During the beta phase, service providers were able to get it operational in less than an hour! This is a game changer if you compare it to the previous more complex stretched cluster solution, that required specialized hardware.

Hardware Independant:  As the replication is completed in software, there is no hardware dependancy and the service provider can utilize the storage of their choice.

Self-Service: Self-service protection, failover, and failback workflows per virtual machine (VM)/vAPP is provided out of the box.

Modern Interface:  Simplified and easy to use modern HTML-5 interface. Further, it offers a Single, unified portal for both tenant and service provider users.

Multi-Tenant Support: vCloud Availability Cloud to Cloud DR 1.0 is fully multi-tenant aware which integrate very well with the strong multi-tenanacy of vCloud Director

Symmetrical replication flow that can be started from either the source or the recovery vCloud Director site.The… Read More

Discovering Vembu BDR Suite v3.6.0

Vembu has recently released v3.6.0 of their Vembu BDR Suite. I was provided an online demo of the product, as well a free trial to try it out in my home lab. Below I summarize my experience with the product and the features I like as well things I thought missing.

Installation Process:

Let’s start by reviewing my experience with the installation process. While Vembu can supply a Vembu BDR virtual appliance to their clients as a method to install their software upon request, the virtual appliance has limited features when compared to the installable version. Further it’s not a full fledge appliance, as would still have to be manage as a Linux machine rather than through a management interface.

For the previously mentioned reasons, I have decided to go with the installable version of Vembu over the appliance version although I usually prefer to use virtual appliances when offered. Most customers will be recommended to use the installable version, which is another reason to go this route. I am hoping this will change in the future, where the virtual appliance will be more mainstream.

While it’s not an appliance, the installation of Vembu BDR suite v3.6.0 was pretty easy to install.… Read More

vCloud Air DR to Cloud with vRealize Automation

Traditionally organizations have reserved their Disaster Recovery capabilities to their most critical workload leaving some important workloads without proper disaster recovery due to the high cost of disaster recovery.

Imagine if you went shopping to buy a car with a friend. When you hit the dealership, your friend tell you that you should buy two cars. The first one will be the one you will be using, where the second one is only for you to use if your first car break? Do you find that as an odd advice or at least too costy one? Was not that exactly what you are doing when building your Disaster Recovery Datacenter? You build a full datacenter and fully pay for it and its compute, storage, cooling, space, and operation 24/7.  Luckily, you did not have a disaster this year, but does not that mean you just paid tons of cash for compute, etc that you never used?

What if you can get the same protection, but almost at the cost of only your storage? That’s what DR to Cloud allow you to do. In fact, vCloud Air DR to Cloud offering is great at that, as the cost you are normally paying for storage covers the cost of protecting your VMs in vCloud Air Disaster Recovery.… Read More

No Excuse for not having backup!

If you are working in the SMB market, you might be facing with the usual challenge of having a very limited to no budget for backup. Please don’t take that as an excuse to skip backup all together, as your management will not tolerate not having backup when things go south. It’s important to recognize that Virtualization availability features while help improve your availability should not be considered a replacement to backup.

The good news, there are quite a few good SMB focused free backup solutions for virtualized environments. In this post, I will be covering one of those options by Unitrends that is not only free, but offer some advanced features that you can usually only find in paid versions of other solutions like Instant VM Recovery.

Unitrends LogoYou will be happy to know that an advanced free backup solution such as “Unitrends Free”, does not require advanced knowledge or training to run. The installer will guide you through the installation and deploy the virtual appliance for you and get you ready in no time and ensure you get it right the first time.

Below are the top features of Unitrends Free:

  • Deployed as a Virtual Appliance – This not only save you on OS licensing, but as well streamline the installation process and ensure a consistent experience for customers.
Read More

Myth busted: vCenter Site Recovery Manager using vSphere Replication for Datacenter migration causes data loss

It seems there is a lot of confusion out there on how vCenter Site Recovery manager work with vSphere Replication when used for a data center migration, thanks in big part to competing products FUDs. Many admins still believe that using vSphere Replication with vCenter Site Recovery Manager for data center migration you will still lose up to 15 minutes of Data. This confusion has evolved due to the following two limitations of vSphere Replication:

  • Lowest RPO possible using vSphere replication  is 15 minutes
  • You cannot replicate powered-off virtual machines. Replication begins when the virtual machine is powered on. You cannot use vSphere Replication to replicate virtual machines templates. <== This statement right of the vSphere documentation.

Here is how the confusion came to life. If you have experienced or read about vCenter Site Recover Manager with storage replication and looked at the sequence of events when doing a data center migration, you will notice it will do a final sync of the data between the two sites right before it cut the replication between the the two sites. If you try to compare the same method with what is happening in vSphere replication and knowing the above two limitations, you will think that when the Data Center Migration is initiated in SRM, it will shutdown the VM and at that time the VM replication was lagging with up to 15 minutes behind based on the provided RPO and as vSphere replication can not replicate after the VM is turned off, the VM will be losing up to 15 minutes of data when coming on the other site, but that is not true as its missing a very minor but important detail that many people seems to over look.… Read More

Unitrends Virtual Backup 8.0 Sneak Peek

I have participated in the Unitrends Virtual Backup 8.0 (formerly PHD Virtual Backup) beta program, and Unitrends has been kind enough to allow me to share my finding with you on my blog. Please note while the screenshots below still show PHDVirtual logo, I believe these will be replaced with the Unitrends logo in the official release.

I have downloaded the beta and the first thing that has impressed me is their new setup wizard. It is really slick to say the least, just for those in doubt, below is the screen shots of the setup wizard while trying to deploy it for the first time in my environment.

The below screenshot are an actual setup steps that I took from my lab environment and not from Unitrends website or documentation as I can see these setup wizard simulating many power point slides I have seen before for how descriptive they are. Well done Unitrends!

Unitrends Virtual Backup Wizard

This is the first screen I was welcomed with as soon I pointed my browser to the appliance. Yes, that was an instructional video that explain the PHD Virtual topology in a very quick manner and here is the video for those of you curious to see what that video was about.… Read More

Chance to Win Google Glass from Veeam

Veeam DR SessionsAs usual Veeam not only offering great products, but they have a great marketing team that always spoil us with new type of  give away. This time they are offering Google Glass for the winner of their three sessions below attendees, and there will be 15 Pizza coupons per session. That means you will have a high odd of winning something just for attending their session and gaining knowledge of their product. Below is the invite, & hope to see you enjoying Google Glass soon!

A long time ago in a galaxy far, far away…

It is a period of data recovery wars. Join us in May for three episodes of epic Whiteboard Fridays: Recovery Wars saga and become true Virtualization Jedi!

Master your skills during these episodes:

• May 13. Episode IV – A New Host: learn how to prepare for disaster, document your environment, ensure your backups and configure backup

• May 20. Episode V – The Admin Strikes Back: be prepared for recover process of the datacenter and primary VM’s, learn how to rebuild your environment effectively

• May 27. Episode VI – Return of the Admin: deep dive into restoring services, best practices around restores, focus on the getting services up and running for the end-users.… Read More

vSphere Data Protection Advanced Licensing your hosts

I have seen a consistent growth lately in the number of my customers that shift to using vSphere Data Protection (VDP). I guess with VDP standard edition being a freebie nonetheless all the powerful features it come up with has made it an easy choice for many customers in the SMB market. VDP standard comes loaded with many features including: (Find out more about each of these features at: http://www.vmware.com/ca/en/products/vsphere/features/data-protection.html )

  • Variable-length de-duplication
  • Global de-duplication
  • Changed Block Tracking backup (CBT)
  • Changed Block Tracking restore
  • One-step recovery

Though as with every free thing in life it comes with few limitations, where most of them are waived when upgrading to the paid version called VDP Advanced. Below is the main limits of the free VDP Standard Edition:

  • Each VDP appliance supports backup for up to 100 virtual machines..
  • Support a maximum of 2 TB of deduplicated backup data.
  • Each vCenter Server can support up to 10 VDP appliances.

Upgrading to the vSphere Data Protection Advanced Edition waive most of these limits as below:

  • Each VDP Advanced supports backup for up to 4,000 VMs in comparison to only 100 in the Standard Edition
  • vSphere Data Protection Advanced expand the maximum supported deduplicated backup datastore to 8TB from 2TB in the standard edition
  • vSphere Data Protection Advanced still limited to 10 VDP appliances per vCenter.
Read More

vSphere Data Protection error – operation failed due to existing snapshot

After I have helped one of my customers deploy VMware vSphere Data Protection Advanced to replace their old VMware Data Recovery, and helped them schedule their environment for backup, we came back the next day to find out that few VMs have failed with the following error: “VDP: operation failed due to existing snapshot”.


As most of our VMs backed up successfully we were pretty sure it is not the VDP software, and that the problem is related to these specific VMs. It end up just like the error is indicating these VMs have Snapshots that have not properly been removed and needed to be cleaned up before VDP can backup these VMs. Below is the action we have taken to fix this problem:

  1. Check the Snapshot Manager for VMs that failed to backup.
  2. If Snapshot Manager showing any snapshots delete them, then proceed to step 4.
  3. If Snapshot Manager not showing any snapshots, just proceed to step 4.
  4. After deleting all snapshots showing in snap manager, create a new snapshot.
  5. Hit Delete All in Snapshot Manager after the new snapshot have been completed.

The above procedure in most cases should clean up your VM from none consistent dirty snapshots, and you should be able to backup these VMs after that using VMware VDP.… Read More

Qtree SnapMirror warnings and limitations with SRM

While setting up SRM with NetApp 6290 at a customer site, my customer was using Qtrees and Qtree SnapMirror which caused us few issues. If you are setting up SRM and using NetApp Qtree SnapMirror, there is quite few warnings, limitations, and best practices that you will need to be aware of. I have listed the most common ones below, though for a more complete list you should check the following document: http://www.netapp.com/us/media/tr-4064.pdf

– Avoid using hidden Qtrees as that seems to cause problems with several versions of the NetApp SRM SRA. One of the most common errors caused by such configuration is:

Error: Failed to sync data on replica device ‘/vol/volume_name/lun#. Device found is neither of SAN type nor of the NAS type. Ensure that the device exists on the storage array and is of type NAS or SAN. 

RM NetApp Hidden Qtree error

– If you have configured qtrees as NFS datastores, you must create an NFS export for each qtree in order for SRM to be able to discover the NFS datastore. If you export only the volume that contains the qtrees, so that there is only one export line for the volume in the /etc/exports file, SRM will not be  able to discover the qtree NFS datastores.
Read More

SRM Error: XmlValidateException Element SourceDevice is not vaild for content model: Source Device.

While setting up vCenter Site Recovery Manager 5.0.3 with NetAPP 6290 filer,  & in particular at the stage where I try to enable the array pair and after I fixed the timeout error I have documented in the following post: SRM Time Out (300 seconds) while waiting for SRA to complete discoverDevice Command, I have faced this error:

Internal error: std::exception ‘class Dr::Xml::XmlValidateException’ “Element ‘SourceDevices’ is not valid for content model: ‘(SourceDevice,)”‘.

A screen capture of the error is below:


SRM NetApp 6290 element SourceDevice is not valid for content model

After a bit of digging I have found out that is a well known NetApp SRA 2.0.1 bug which is documentd here, and there is few possible workaround. Below I will document these workarounds, where you can take the most feasible option for your environment.

1- Revert back to NetApp SRA 2.0.0 as that does not suffer the same problem. This bug only seems to affect NetApp SRA 2.0.1 not 2.0.0. This is actually the solution I went with as it required the least amount of work and maintenance going forward.

2- Download and install NetApp SRA Patch 2.0.1P2. This patch does not seems to be generally available and you will have to request it from NetApp support or digg for it into the NetApp support portal.… Read More

SRM Time Out (300 seconds) while waiting for SRA to complete discoverDevice Command

While setting up vCenter Site Recovery Manager 5.0.3 with NetAPP storage,  & in particular at the stage where I try to enable the array pair I was welcomed with the following error:

SRM Time Out (300 seconds) while waiting for SRA to complete ‘discoverDevice’ Command.

 SRM Time out while waiting for SRA to complete discoverdevice command


This SRM SRA Adapter error is not limited to NetApp storage, but a similar error can be seen with EMC & other vendors storage. If your storage is pretty busy or you have too many devices on it, the operation just can not get completed within the 300 Second (5 minutes) time out set by default in SRM. The solution for this problem is really simple, all you have to do is to increase the storage.commandTimeout . In my case, I just increased it to 1200 second (20 minutes) which should be more than sufficient for most environments.  Further, it is important to mention that this time out setting is setup per site, & must be repeated for both sites involved in your SRM setup. Below is the exact instruction:

1. Click Sites in the left pane, right-click your primary site, and click Advanced Settings.

2. In the navigation pane of the Advanced Settings window, click Storage.… Read More

VMware vCenter Site Recovery Manager Service failed to start

While installing VMware SRM 5.0.3 at a customer site, both nodes installations have given me the following error right before the installation completion:

Failed to start service.

VMware vCenter Site Recovery Manager Service failed to start.
Check that all required Windows services are running. View the server log for more information.

Press Retry to try again or press Cancel to exit installation.

For those of you who want to see the actual error as it show on screen, below is a screenshot of the error:

SRM Service failed to start error

It turned out the solution is really simple, just change the VMware Site Recovery Manager Service in Windows Service Manager to use the domain service account you used during the SRM installation rather than the local account and then try to start the service again. The service should start up and you should be able to hit the retry button on the installation wizard where the installation of Site Recovery Manager complete successfully after.

I am not sure for whatever reason during VMware SRM setup, the VMware Site Recovery Manager service got installed using local system rather than the service account entered during the installation. I thought to share this quick fix with anyone tumbling through it.… Read More