Assign Domain Users vCenter Single Sign Administrator Privileges

If you have upgraded your Virtual Infrastructure to vSphere 5.1 or 5.5, you are already aware of the addition of Single Sign On. When installing Single Sign On, a default user Admin@System-Domain is created for you with a password that you have assigned to it during the vCenter Single Sign On installation. The Admin@System-Domain user is initially the only single user that have access to manage your the Single Sign On portion of your vSphere environment.

As a good practice & to be able to track who is responsible for a particular change in your SSO you might want to assign your vSphere Administration team domain accounts administrator privillages to your vCenter Single Sign On. While it is an easy task to do, the way it must be done seems to confuse many admins who is new to vCenter SSO. Below is the instruction of doing so:

  1. Browse your vSphere Web Client Portal (https://<Your vCenter Machine>:9443).  (Note: SSO only can be managed using the Web Client)
  2. Browse to Administration > Access > SSO Users and Groups in the vSphere Web Client.
  3. Click on the Groups Tab
  4. Click on the desired Group (_Administrators_)
  5. Click Add Principals (The icon of a person with a plus sign next to it highlighted in the above screenshot)
  6. Select the identity source that contains the principal to add to the group (Probably your Domain)
  7. Search for the desired user
  8. Select the desired user and click Add
  9. Repeat step 6-8 to add the rest of the desired users.
Read More

ATS-Only VMFS Volume ‘VMFS5’ not mounted. Host does not support ATS or ATS initialization has failed.

While being at a customer site last week, I was asked if I could help with a mysterious VMFS datastore behavior. That particular datastore out of a sudden became none accessible and they could not carry out any changes to it. They can not VMotion in or out of it, or even create a folder into it. After running Storage Adapters Rescan on some of the ESXi hosts, they could not view that datastore any more. Checking out the logs at /var/log/vmkernel.log, we have noticed the error “ATS-Only VMFS Volume ‘VMFS5′ not mounted. Host does not support ATS or ATS initialization has failed.” shown in the below screenshot (Double Click it for full size).

ATS-Only VMFS Volume ‘VMFS5′ not mounted. Host does not support ATS or ATS initialization has failed.

What is Atomic Test & Set (ATS)?

Before I go about explaining the cause of the above error & how to resolve it, I thought it will make  sense to share some background about ATS and where is the idea of ATS-Only VMFS Volumes has came from which directly relate to the cause of this problem.

Atomic Test & Set (ATS) was introduced as one of the fundamentals operations of vStorage API Array Integration(VAAI). ATS is used during creation and locking of files on the VMFS volume.… Read More

How to replace vCenter 5.1, SSO, Web Client, vCO Certificates

With the release of vSphere 5.1 certificates started to play a much more vital role, where having invalid certificates in your environment is not an option anymore as it could break the operation of your environment as well forbid you from logging in. This change has been done to increase the security of your Virtual Infrastructure Management Components (vCenter Service, Inventory Services, SSO, Web Client, vCO, Update Manager, & vCenter Log Browser) & to compact the possibilities of man in the middle attacks. This change has brought a lot of challenges to many VMware customers who had invalid and expired certificates in their environment without even noticing it. The tedious process of replacing any of these certificates have not been a pleasure work for many, the good news is that VMware has just released vCenter Certificate Automation Tool 1.0 to streamline the process & release much of that pain.

VMware has just announced the general availability of vCenter Certificate Automation Tool 1.0. This tool provides an automated mechanism to replace certificates in the following components of the vCenter management platform:

  • vCenter Server
  • vCenter Single Sign On
  • vCenter Inventory Service
  • vSphere Web Client
  • vCenter Log Browser
  • vCenter Orchestrator (VCO)
  • vSphere Update Manager (VUM)

The tool can be downloaded for free from: More

vCenter Single Sign On 5.1 best practices

Since vCenter Single Sign On was introduced in vSphere 5.1, many questions have been rising around it. There seems to be a very limited amount of resources out there that document best practices related to vCenter Single Sign On, which is the reason for me to develop this post where I will try to combine as many best practices and answers related to vCenter 5.1 Single Sign On as possible.

I have been one of the lucky consultants who has already got to design/implement vSphere 5.1 for quite few enterprise customers where I have got to debate and drive best practices that I used across those implementations. I am sharing them here where others can benefit from them as well to allow a room for others to debate them and contribute their feedback.

Where to install vCenter Single Sign On (Physical vs Virtual)?

Just as the recommendations have always been for vCenter using virtual machine(s) is the best practice to save on cost and benefit of the availability features built in vSphere, that is no difference in vSphere 5.1. You can host all vCenter 5.1 components including SSO on virtual or physical machine, where virtual machine is the recommended practice due to the same reason mentioned earlier.… Read More

Call “HostDatastoreSystem.CreateVmfsDatastore” for object “ha-datastoresystem” on ESXi “” failed.

While working with my home vSphere 5.1 lab the other day, I was trying to create a VMFS5 datastores on my local SATA disk. Each time I tried to do that I was just welcomed with the following error:

Call “HostDatastoreSystem.CreateVmfsDatastore” for object “ha-datastoresystem” on ESXi “” failed.

Please note stand for my host IP. To visualize the error below is a screen shot of the error as it has appeared in my home lab

Call "HostDatastoreSystem.CreateVmfsDatastore" for object "ha-datastoresystem" on ESXi "" failed.

After fuzzing around trying to figure out what happened, I have remembered this particular disk was used by one of my old lab ESXi hosts. As I do all kind of crazy things in my labs, I thought I should try to wipe the disk clean then try to format it with VMFS5 afterward. That has actually solved the problem. This error seems to happen if you have a file system on that LUN/disk that ESXi does not understand, cannot overwrite or if You don’t have a full write access to the disk/LUN(ex: SRM replication target)

Below is how I went about doing it if you are not familiar with the procedure:

Note: In previous versions of ESXi fdisk was your friend in such a situation, though if you try it in vSphere 5.x… Read More

vSphere 5.1 VMware Tools NTP Settings

In earlier versions of VMware vSphere, many of us used to configure VMware tools settings by double clicking VMware tools inside the guest OS. This was most often used to configure NTP in the Virtual Machine VMware tools. Just to remind you of what that looked like:

VMware Tools Properties

Alright if you try to do the same in vSphere 5.1, you will be surprised that you will not have any options to choose when double clicking on VMware tools inside the guest OS. What you will get will look like the below screen shot:

vSphere 5.1 VMware tools settings no NTP check box

Maybe what I have mentioned so far is already what you know and its why you got here. Now let’s cover how you can configure your VMware Tools to sync the VM time with your host. The way this is done in vSphere 5.1 is a bit different. You actually don’t need to login to the VM to change VMware tools setting & in most use cases the VMware Tools NTP Sync setting, but now you can do it from the VMware setting page. The VMware tools settings now can be accessed  in Virtual Machine –> Edit Settings –> Options –> VMware Tools. The below screenshot show just how to set it up.… Read More

Microsoft Exchange 2010 is definetly supported on VMware vSphere

I know most of my readers are already aware that Microsoft Exchange & Microsoft SQL have been supported on VMware vSphere for quite long time. In fact, there is so many companies are using it at unbelievably large production scale. Though today while going back from my customer site by train, I was talking to two of their engineers. One of them worked within their Virtual Infrastructure team & the other one within their Microsoft Infrastructure team. What sparked the idea of this post is when the Virtual Infrastructure engineer asked his colleagues what he think of virtualizing their Exchange 2010 setup. The shocking answer was “Our Exchange 2010 environment is too large to be virtualized as we have about 10,000 users. Further, if we virtualize we have have to use Hyper-V to be supported by Microsoft although I know it will run better on VMware vSphere.”

Let me address the status of Microsoft support for running MS Exchange 2010 on VMware vSphere, as it seems there is a lot of misleading believe that Microsoft Exchange 2010 is only supported on Hyper-v. Unfortunately, this seems to be fueled even further by few Microsoft Sales Reps that does not play fair and welling to do anything to win a deal even by misrepresenting facts.… Read More

Data Center Virtualization

Companies once prided themselves on having a huge data center full of servers, storage devices and network hubs, all working together to run the business. You need a huge amount of infrastructure just to support your computing infrastructure. One mammoth search engine company owns a data center that is so complex that it even creates its own climate within the building.

Data Center Virtualization is about separating hardware, software and networks so they no longer have to physically co-exist. It’s also about combining functionality and sharing resources to reduce the amount of hardware needed. This leads to less license fees, less maintenance and less overall infrastructure overhead.

Virtualization of data center functionality can happen at various levels of the architecture. The result could be a completely virtual architecture or one that combines virtual and dedicated functions.

Virtualization of Operating Systems

Instead of a machine dedicated to one particular operating system (OS), virtual machine managers (VMMs) can control multiple OS running on one machine. Each OS runs as if it was the only one on that machine. Users log into their computer as usual and the VMMs direct their request to the appropriate OS. The multiple OS instances don’t communicate with each other so there’s no fear that a Windows 8 user will get a response from the Redhat Linux OS.… Read More

vCenter 5.1 Installation(Part 5) – vSphere Web Client Step by Step

Alright now that you got your vCenter 5.1 up and running & ready to start managing it. I know vSphere Client will be the first thing to come to your mind in here, but its worth mentioning that all the new features in vSphere 5.1 is only included in the vSphere Web Client not the traditional Installable vSphere Client. Alright that should get you enough reason to install and try to get used to the new vSphere Web Client. Though the new vSphere Web Client has been improved dramatically from the one included in vSphere 5.0 that it feels it is a fully different client. It is much faster, smoother and with tons more functionality that can replace almost every functionality in the traditional vSphere Client.

While this post show you how to install the vSphere Web Client in a step by step fashion, if you have not yet setup vCenter 5.1 then you might want to look at previous posts in this series which document vCenter 5.1 installation including preparing the DBs.

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

vCenter 5.1Read More

vCenter 5.1 Installation(Part 4) – vCenter Service Step by Step

As covered in my previous three posts, vCenter Service is the third component to be installed. As a reminder the order of installing vCenter 5.1 components is as follow:

Single Sign On ==> vCenter inventory Service ==> vCenter Service.

In this post, I will demonstrate in a step by step fashion how to install the vCenter Service though if you have not followed earlier parts in this series you will need to check them out before you install the vCenter Service. The earlier posts in this series can be found at:

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

Alright so now that you have completed the installation of SSO and Inventory Service, you are ready to start the installation of vCenter 5.1 Service & below is a step by step instruction on how to do just that.

To install vCenter Server

1. Launch the installer using an account with administrator privileges.

2. Select vCenter Server from the VMware Product Installers menu and click Install.

vCenter 5.1 Server Installation Wizard

3. Select the setup language and click OK.

Select vCenter 5.1 Setup Language

4. Wait while the installation process begins.

Wait for VMware vCenter Server 5.1 installation process to begin

5.… Read More

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

In my previous two posts, I have demonstrated how to prepare the databases required for the different vCenter 5.1 components(SSO, vCenter Service, & Update Manager)  as well how to install vCenter Single Sign On. If you have not went through these earlier two posts, then you will need to follow them before proceeding with this one. These two posts can be found at:

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

As I have mentioned in my earlier post, the next vCenter 5.1 component to install would be vCenter Inventory Service. In this post, I will demonstrate how to install the vCenter Inventory Service in a step by step fashion. It is important to note that in vCenter 5.1 you have the option to install the vCenter Inventory Service with other vCenter components or on a different server/vm. As I mentioned in my first post in the series, the main reason why sometime you want to install it on a separate VM/Server is if scalability is a concern in your organization and you are approaching the vCenter Scalability limits of 1,000 hosts and 10,000 VMs. In most organizations, where these limits are not even close then installing the vCenter Inventory Server on the same VM/Server running the vCenter Service is a no brainer.… Read More

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

During the installation of vCenter 5.1, you will need to install 3 components in the following order: Single Sign On => Inventory Service => vCenter Service. In a new installation I would normally install the Web Service after installing the vCenter Service, though during an upgrade I would install the web service right after the Single Sign On service to be able to use it just in case I wanted to check on my Single Sign On configuration or want to troubleshoot. As this guide assuming a new installation we will leave the Web Client Service to the end. In this post, I will demonstrate the installation of the Single Sign On Service.

Preparing Databases for vCenter Components

Three vCenter components require a database. Single Sign On, vCenter Service, & Update Manager each of those components require its own database, where the creation of those databases have been documented at the first post in this series found at: vCenter 5.1 Installation(Part 1) – Preparing the Databases.

Alright now you have your databases ready let’s start the process of installing vCenter Components. The first component to install as mentioned earlier is the Single Sign On Service, which is documented in a step by step fashion below.… Read More

vCenter 5.1 Installation(Part 1) – Preparing the Databases

After the introduction of vSphere 5.1, there seemed to be a lot of fuzz about the installation of the new vCenter components. I believe most of the hype was caused about how the initial vSphere 5.1 release behaved differently against expired certificates from how vSphere version prior to 5.1 behaved. In earlier releases, vCenter has only checked the expiry date of the certificate used during the initial install and fall to a backup mechanism if the certificate fail though the service would went up and the user would use vCenter as nothing has happened. To increase the security of vCenter and prevent man in the middle attacks, this behavior was changed in vCenter 5.1. vCenter 5.1 is always checking the validity of its certificates every time the service is being started & it would report an error if it does not find a valid certificate. As many customers had an expired vCenter certificates and did not know about it before upgrading to vSphere 5.1 they were caught off guard by this small behavior change where VMware has quickly released a quick workaround for it and a new patch were released to improve how vCenter response to this behavior.

The installation of vCenter 5.1… Read More

VMware vSphere Free Compliance Checkers

While data security has always been one of the top CEOs concerns for the past decade or so, more compliance regulation has been hitting organizations doors every year. After organizations have mastered how to secure their physical environment by using physical segregation, Virtualization came around and changed the security game. Now organizations has to ensure that their Virtual Infrastructures are meeting compliance regulations such as PCI, HIPAA, SOX, and FISMA. Trying to manually keep up with such regulations that get updated regularly can be an endless hassle, & that why VMware has released vCenter Configuration Manager few years back.

vCenter Configuration Manager can help organizations not only audit their virtual infrastructure against compliance regulations such as Sarbanes-Oxley (SOX), Payment Card Industry (PCI), Health Insurance Portability and Accountability Act (HIPAA), and Federal information Security Act (FISMA), but as well it help you audit your environment against vSphere best practices including vSphere Security best practices. vCenter Configuration Manager can even help you re-mediate your environment to ensure it is meeting such regulation where many of those remediation can be carried out automatically others will include the recommendations on how to fix it. vCenter Configuration Manager is not limited to the vSphere Infrastructure, but it can check physical machines, Windows/Linux/Unix OS, & even applications for such compliance.… Read More

VMware vSphere 5.1 new vCenter architecture & Single Sign on

With the introduction of vSphere 5.1, VMware has introduced a new vCenter architecture & SSO ( Single Sign On). This is seems to be the first thing being noticed by customers when deploying vSphere 5.1 as its not exactly what they used to. Prior to vSphere 5.1 all vCenter services were installed at once on a single server without giving you the option of spreading them across multiple servers or not installing certain services. To allow vCenter to scale even further, in vSphere 5.1 you got the option to install four separate services that constitute the vCeter Server 5.1 platform. These are:

• vCenter Single Sign On (SSO)
• vCenter Inventory Service
• vCenter Server
• vSphere Web Client

It is important to understand that you can install all those services on the same machine/VM without any problem, & that what is actually being done if you choose the simple install when installing vCenter. If your environment setup is made of a single vCenter then this simple setup would be more than adequate for your environment, and you have no reason to split these components up.  On the other hand, if you have multiple vCenters in your environment and its larger more complex environment then you are better off installing those components on different machines by using their separate installation links for a better scalability.… Read More