vCenter Appliance fsck failed

While I have been working with a customer on upgrading their infrastructure to vSphere 5.5, they had a power outage during the night which got their full virtual infrastructure down including storage (Not sure what the story with their UPS/Power generator). After they have brought their virtual infrastructure backup up both their vCenter Appliance and VDP appliance and few other Linux VMs has failed the fsck check while it was booting and it was stuck at the following error:


fsck failed.  Please repair manually and reboot.  The root
file system is currently mounted read-only.  To remount it
read-write do:

bash# mount -n -o remount,rw /

Atention:  Only CONTROL-D will reboot the system in this
maintanance mode.  shutdow or reboot will not work.

Give root password for maintenance
(or type Control-D to continue):


Below is an actual screenshot of the error for you to compare it with what you got:

VMware vCSA fsck failed

Below is the steps I have followed to fix this on both vCSA and VDP Advanced:

  1. Stay Calm & Don’t Panic!
  2. Take a backup/snapshot in case the below steps does not work for you and you need to revert it back to the current state when you call support or call for help.
  3. Read More

vCenter Single Sign On 5.5 Whats New

vCenter Single Sign On has a considerable amount of changes in vSphere 5.5, with few major ones. Many of these changes have went undetected or unnoticed by the Virtual Infrastructure Admins. If you have deployed vSphere 5.5 and missed these changes or planning to install vCenter SSO 5.5 and want to learn what has changed from the vSphere 5.1 days, then this post is for you:

Below is the list of the major changes introduced in vCenter Single Sign On:

 vCenter SSO Architecture Improvements:

1- Multi master: Unlike 5.1, vCenter SSO 5.5 has A built-in automatic multi-master replication architecture that ensure that all SSO instances are always kept in sync. While this sound great, most admins are not sure what it means nor how it affect the way they design SSO. To understand the value of this change, you must understand how SSO worked in 5.1. and how that changed in 5.5.

In 5.1 if you wanted to enable SSO for multiple vCenters in your environment, you needed to point all of them to the same SSO instance which made all of those vCenters dependent on that single SSO instance. This has made that single SSO instance to be crucial for the operation of all of your vCenters, as if that SSO instance goes down you will not be able to access any of your vCenters.… Read More

vCenter Server Appliance Error: failed to connect to VMware Lookup Service

While working with VMware vCenter Server Appliance 5.5, I needed to change the IP Address of the appliance as my customer were changing their IP scheme. It was quite to go to the VMware vCSA then to the network tab then Address tab and change the IP Address. After rebooting the appliance to ensure the IP change has taken affect and is working properly and consistent, I was able to login to the vCenter Server Appliance Management portal, but I was not able to connect to using the Web Client and was kept getting the following error:

“Failed to connect to VMware Lookup Service https://<vCenter Appliance IP>/lookupservice/sdk – SSL certificate verification failed”

Or as shown in the below screenshot:

Web Client Error after changing vCenter Appliance IP


It seems changing the IP Address of the vCenter Server Appliance does not regenerate the certificate with the new IP and you will have to configure the appliance to generate a new one manually. The good news it is pretty easy to do, and below is the instruction:

  1. Log in to the vCenter Server Appliance Web interface at https://<vCSA IP>:5480/.
  2. Click the Admin tab.
  3. Regenerate certificates:
    • vCenter Server 5.1: Select Toggle certificate setting so that the Certificate regeneration enabled displays Yes.
  4. Read More

vCenter Server Appliance 5.5 limitations

VMware vCenter Server Appliance 5.5 has been gaining huge attraction lately, especially with many of the major limitations affecting the vCenter Appliance in previous verions have been lifted. This is including the limitation where it was only supposed to support 5 hosts and 50 VMs when using the embedded database. In fact the all shiny new VMware vCSA 5.5 support up to 100 hosts and 3000 VMs using the embedded database, which seems to be able to deliver up to the scale required by most customers.

Note: It seems a misleading assumption has been going around that 100 hosts and 3,000 VMs is the maximum supported configuration by VMware vCSA 5.5, where is that is absolutely not true. This limit only apply when using the embedded  built-in vPostgres where it actually can support up to the vCenter maximums of 1,000 hosts and 10,000 VMs when connected to an external Oracle Database. This is clearly documented on page 7 of vSphere 5.5 Maximums guide and a copy of that table is shown below. I believe this misleading belief has spread as many bloggers has mentioned the 100 hosts and 3,000 VMs limit while failing to point out it is not the actual limit of the appliance, but it only apply when using the embedded database. 

VMware vCenter Server Appliance Maximum Configurations

I have actually been using VMware vCenter Server Appliance 5.5 for most of my labs and at various customers as it deliver what they needs, as it can be deployed in fraction of the time and save them from having to build Windows and DB servers, not to mention its easier to maintain and manage.… Read More

Deploying VMware vCenter 5.5 Appliance

Lately for most of my labs and even for many customers I have been deploying the VMware vCenter 5.5 Appliance, as it is much quicker to install and give less chances for error. Further it saves customers a Windows license and not to mention being much easier to maintain, patch and upgrade. Further, with many of the earlier versions limitations being lifted it becoming much more viable option than ever before. For that I have decided to document the process of installing the VMware vCenter 5.5 Appliance in a hope to encourage more of you to try it out.

 VMware vCenter 5.5 Appliance Installation Steps:

1- Download the VMware vCenter 5.5 Appliance OVA (I rather the OVA as all you have to get and deploy is a single file)

2- Deploy the VMware vCenter 5.5 OVA into one of your hosts.

  1. Launch the vSphere Client or vSphere Web Client.
  2. Go to File > Deploy OVF Template.
  3. Browse to the location of the vCenter Server Appliance downloaded .ova or the .ovf file and click Open.
  4. In the OVF Template Details page, click Next.
  5. In Name and Location, enter the name of your vCenter Server Appliance and click Next.
  6. Read More

Assign Domain Users vCenter Single Sign Administrator Privileges

If you have upgraded your Virtual Infrastructure to vSphere 5.1 or 5.5, you are already aware of the addition of Single Sign On. When installing Single Sign On, a default user Admin@System-Domain is created for you with a password that you have assigned to it during the vCenter Single Sign On installation. The Admin@System-Domain user is initially the only single user that have access to manage your the Single Sign On portion of your vSphere environment.

As a good practice & to be able to track who is responsible for a particular change in your SSO you might want to assign your vSphere Administration team domain accounts administrator privillages to your vCenter Single Sign On. While it is an easy task to do, the way it must be done seems to confuse many admins who is new to vCenter SSO. Below is the instruction of doing so:

  1. Browse your vSphere Web Client Portal (https://<Your vCenter Machine>:9443).  (Note: SSO only can be managed using the Web Client)
  2. Browse to Administration > Access > SSO Users and Groups in the vSphere Web Client.
  3. Click on the Groups Tab
  4. Click on the desired Group (_Administrators_)
  5. Click Add Principals (The icon of a person with a plus sign next to it highlighted in the above screenshot)
  6. Select the identity source that contains the principal to add to the group (Probably your Domain)
  7. Search for the desired user
  8. Select the desired user and click Add
  9. Repeat step 6-8 to add the rest of the desired users.
  10. Read More

vCenter 5.1 Installation(Part 5) – vSphere Web Client Step by Step

Alright now that you got your vCenter 5.1 up and running & ready to start managing it. I know vSphere Client will be the first thing to come to your mind in here, but its worth mentioning that all the new features in vSphere 5.1 is only included in the vSphere Web Client not the traditional Installable vSphere Client. Alright that should get you enough reason to install and try to get used to the new vSphere Web Client. Though the new vSphere Web Client has been improved dramatically from the one included in vSphere 5.0 that it feels it is a fully different client. It is much faster, smoother and with tons more functionality that can replace almost every functionality in the traditional vSphere Client.

While this post show you how to install the vSphere Web Client in a step by step fashion, if you have not yet setup vCenter 5.1 then you might want to look at previous posts in this series which document vCenter 5.1 installation including preparing the DBs.

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

vCenter 5.1 Installation(Part 4) – vCenter Service Step by Step

Alright for those of you ready to install the vSphere 5.1 Web Client,  please find the promised step by step instruction below.… Read More

vCenter 5.1 Installation(Part 4) – vCenter Service Step by Step

As covered in my previous three posts, vCenter Service is the third component to be installed. As a reminder the order of installing vCenter 5.1 components is as follow:

Single Sign On ==> vCenter inventory Service ==> vCenter Service.

In this post, I will demonstrate in a step by step fashion how to install the vCenter Service though if you have not followed earlier parts in this series you will need to check them out before you install the vCenter Service. The earlier posts in this series can be found at:

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

Alright so now that you have completed the installation of SSO and Inventory Service, you are ready to start the installation of vCenter 5.1 Service & below is a step by step instruction on how to do just that.

To install vCenter Server

1. Launch the installer using an account with administrator privileges.

2. Select vCenter Server from the VMware Product Installers menu and click Install.

vCenter 5.1 Server Installation Wizard

3. Select the setup language and click OK.

Select vCenter 5.1 Setup Language

4. Wait while the installation process begins.

Wait for VMware vCenter Server 5.1 installation process to begin

5.… Read More

vCenter 5.1 Installation(Part 3) – vCenter 5.1 Inventory Service Installation

In my previous two posts, I have demonstrated how to prepare the databases required for the different vCenter 5.1 components(SSO, vCenter Service, & Update Manager)  as well how to install vCenter Single Sign On. If you have not went through these earlier two posts, then you will need to follow them before proceeding with this one. These two posts can be found at:

vCenter 5.1 Installation(Part 1) – Preparing the Databases

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

As I have mentioned in my earlier post, the next vCenter 5.1 component to install would be vCenter Inventory Service. In this post, I will demonstrate how to install the vCenter Inventory Service in a step by step fashion. It is important to note that in vCenter 5.1 you have the option to install the vCenter Inventory Service with other vCenter components or on a different server/vm. As I mentioned in my first post in the series, the main reason why sometime you want to install it on a separate VM/Server is if scalability is a concern in your organization and you are approaching the vCenter Scalability limits of 1,000 hosts and 10,000 VMs. In most organizations, where these limits are not even close then installing the vCenter Inventory Server on the same VM/Server running the vCenter Service is a no brainer.… Read More

vCenter 5.1 Installation(Part 2) – Single Sign On Installation

During the installation of vCenter 5.1, you will need to install 3 components in the following order: Single Sign On => Inventory Service => vCenter Service. In a new installation I would normally install the Web Service after installing the vCenter Service, though during an upgrade I would install the web service right after the Single Sign On service to be able to use it just in case I wanted to check on my Single Sign On configuration or want to troubleshoot. As this guide assuming a new installation we will leave the Web Client Service to the end. In this post, I will demonstrate the installation of the Single Sign On Service.

Preparing Databases for vCenter Components

Three vCenter components require a database. Single Sign On, vCenter Service, & Update Manager each of those components require its own database, where the creation of those databases have been documented at the first post in this series found at: vCenter 5.1 Installation(Part 1) – Preparing the Databases.

Alright now you have your databases ready let’s start the process of installing vCenter Components. The first component to install as mentioned earlier is the Single Sign On Service, which is documented in a step by step fashion below.… Read More

vCenter 5.1 Installation(Part 1) – Preparing the Databases

After the introduction of vSphere 5.1, there seemed to be a lot of fuzz about the installation of the new vCenter components. I believe most of the hype was caused about how the initial vSphere 5.1 release behaved differently against expired certificates from how vSphere version prior to 5.1 behaved. In earlier releases, vCenter has only checked the expiry date of the certificate used during the initial install and fall to a backup mechanism if the certificate fail though the service would went up and the user would use vCenter as nothing has happened. To increase the security of vCenter and prevent man in the middle attacks, this behavior was changed in vCenter 5.1. vCenter 5.1 is always checking the validity of its certificates every time the service is being started & it would report an error if it does not find a valid certificate. As many customers had an expired vCenter certificates and did not know about it before upgrading to vSphere 5.1 they were caught off guard by this small behavior change where VMware has quickly released a quick workaround for it and a new patch were released to improve how vCenter response to this behavior.

The installation of vCenter 5.1 has been much smoother after releasing vCenter 5.1b & ESXi 5.1a, & to calm my readers nerve about installing vCenter 5.1, I will be showing in here a step by step the installation process of vCenter 5.1 in a simple way that show its not much more difficult than what used to be done in vSphere 5.0 if you know what you are doing.… Read More

VMware vSphere 5.1 new vCenter architecture & Single Sign on

With the introduction of vSphere 5.1, VMware has introduced a new vCenter architecture & SSO ( Single Sign On). This is seems to be the first thing being noticed by customers when deploying vSphere 5.1 as its not exactly what they used to. Prior to vSphere 5.1 all vCenter services were installed at once on a single server without giving you the option of spreading them across multiple servers or not installing certain services. To allow vCenter to scale even further, in vSphere 5.1 you got the option to install four separate services that constitute the vCeter Server 5.1 platform. These are:

• vCenter Single Sign On (SSO)
• vCenter Inventory Service
• vCenter Server
• vSphere Web Client

It is important to understand that you can install all those services on the same machine/VM without any problem, & that what is actually being done if you choose the simple install when installing vCenter. If your environment setup is made of a single vCenter then this simple setup would be more than adequate for your environment, and you have no reason to split these components up.  On the other hand, if you have multiple vCenters in your environment and its larger more complex environment then you are better off installing those components on different machines by using their separate installation links for a better scalability.… Read More

How to Export your VM Inventory list from vCenter

Time will come when you need a quick inventory of the VMs in your vCenter with their IPs, Computer name, power on status & all these simple things. You might just need that sheet b/c you need to keep an inventory of what you have or just the for the simple cause that the management had asked you for such one. In both cases you need to figure out the quickest and easiest way to do it. Yeah this can be achieved using power shell script or any other scripting language that vSphere support, but let’s keep that to when we require a more extensive inventory that is can not be generated using the GUI of vCenter. One question I repeatedly hear “why is the Export list in vCenter file menu is shadowed and can not be chosen?” Here I will show you the steps to get a simple inventory of your VMs using VMware vCenter GUI & following it will show you how to get the Export list in vCenter file menu in business.

Steps to Export your VM Inventory list from VMware vCenter

1- Open the vSphere Client and connect to your vCenter

2- From the top menu go to View ==> Inventory ==> VMs & Templates

3- From the inventory list on the left side choose the VMs you want to include in your inventory export or choose a top level folder that includes the VMs you want to export

4- Choose the Virtual Machines tab from the the tabs on the top of the right side.… Read More

VMware vCenter Server Appliance Error: VPXD must be stopped to perform this operation.

While playing with VMware vCenter Server Appliance in my home lab, I have been surprised with the following error:

Error: VPXD must be stopped to perform this operation.

This error has came up while trying to change authentication to active directory authentication and kinda seen the same error when trying to change the database to an external database. For some reason the Server service stop button is shadowed as well which mean I could not stop it by the GUI interface. Although my lab setup is not ideal and it might just due to the way I set it up, though I will still document how I resolved this where the same mechanism can be used to stop and restart any other service used by the vCenter Server Appliance. Below are the steps I have followed:

1- SSH to your VMware vCenter Server Appliance using the root account.

2- Execute the following command to see the status of all the service running in the vCenter Appliance:   chkconfig

The output of all services will look something like below:

localhost:~ # chkconfig
after.local               off
apache2                  off
arpd                         off
atftpd                       off
auditd                       on
autoyast                    off
chargen                      off
chargen-udp              off
cron                       on
daytime                    off
daytime-udp              off
dbus                     on
dcerpcd                  on
dhcp6r                   off
dhcp6s                   off
dhcpd                    off
discard                  off
discard-udp              off
earlysyslog              on
echo                     off
echo-udp                 off
eventlogd                on
fbset                     on
gpm                      off
haldaemon                on
haveged                  on
irq_balancer             on
kbd                      on
ldap                     on
lsassd                   off
lwiod                    on
mdadmd                   off
multipathd               off
netlogond                on
netstat                  off
network                  on
network-remotefs         on
nfs                      on
ntp                      off
pcscd                  off
powerd               off
random               on
raw                      off
rpasswdd            off
rpcbind                on
rpmconfigcheck           off
sendmail                       on
servers                           off
services                         off
setserial                        off
skeleton.compat          off
splash                            on
splash_early                on
sshd                               235
stunnel                         off
syslog                           on
syslog-collector         off
systat                          off
time                            off
time-udp                   off
uuidd                         off
vami-lighttp             235
vami-sfcb                 235
vaos                          235
vmware-inventoryservice  on
vmware-logbrowser        off
vmware-netdumper         off
vmware-rbd-watchdog      off
vmware-tools             on
vmware-vpostgres         on
vmware-vpxd              on
vsphere-client           on
xinetd                   off
ypbind                   off

3- Stop the required service in my case was vmware-vpxd using the following command: chkconfig service-name off (ex: chkconfig vmware-vpxd off)

4- carry out your changes

5- Start the service again using the following command: chkconfig service-name on (ex: chkconfig vmware-vpxd on)

I hope this help someone out there from being stuck with such issue… Read More

vCenter to Manager Hyper-V is a reality

Working for VMware has always been filled with surprises since the first day I joined. I really believe we have one of the greatest engineering departments out there. I mean when your engineering team create a plugin that allow vCenter to Manager Hyper-V in their own spare time, it just get you really proud to work for such a company. Before I get into details about what is going on, I must thank our great engineering team for their super dedication.

VMware has finally released its Hyper-V management tool as a “fling” (unsupported vCenter Plug in, personal projects built by VMware Engineers in their spare time), now available at:

Here is what our Engineering Team have to say about it:

VMware vCenter XVP Manager and Converter provides basic virtualization management capabilities for non-vSphere hypervisor platforms towards enabling centralized visibility and control across heterogeneous virtual infrastructures. It also simplifies and enables easy migrations of virtual machines from non-vSphere virtualization platforms to VMware vSphere.


  • Management of the following Microsoft Hyper-V platforms:
    • Microsoft Hyper-V Server 2008
    • Microsoft Windows Server 2008 (64-bit) with Hyper-V role enabled
    • Microsoft Hyper-V Server 2008 R2
    • Microsoft Windows Server 2008 R2 with Hyper-V role enabled
  • Familiar vCenter Server graphical user interface for navigating through and managing non-vSphere inventory
  • Ease of virtual machine migrations from non-vSphere hosts to vSphere inventory (Great
  • Compatible with VMware vCenter Server 4.0 & 4.1
  • Scalable up to management of 50 non-vSphere hosts

Again much more info & videos can be found at:

Do you think this should be a supported feature?… Read More