What’s new in NSX Migration for VMware Cloud Director 1.2

As most of you are aware by now, VMware announced the sunsetting of NSX for vSphere (NSX-V), and the current end of general support is targeted for January 2022, while the end of technical guidance will be in January 2023. It is important that Cloud Providers migrate from NSX-V to NSX-T as soon as possible. As the process for larger providers will take a good amount of prep work and testing, the earlier they start the better their experience will be. The good news, the migration tools available this time is very helpful to streamline the process.

There are multiple methods to migrate from NSX for vSphere to NSX-T Data Center. They are listed below in the order of their usability in a VMware Cloud Director environment:

  • VMware NSX Migration for VMware Cloud Director can migrate the workload VMs and other organization VDC objects to the same vCenter Server instance managed by VMware Cloud Director. This is the option you really want to use if you have VMware Cloud Director in your environment. Here is the link for download. Here is the link as well for features supported by the migration tool.
  • Coexistence – New workloads are deployed on NSX-T, and the older workloads are allowed to die over time.
Read More

How to Change VMware NSX-T Manager IP Address

VMware NSX-T logoThere is often the situation where you need to change the IP addresses of your NSX-T Managers. For example, you might be changing your IP schema as I am doing currently in my home lab. NSX-T does not have a field to change the IP address of it’s NSX Managers, but you will need to add new NSX Managers with the new desired IP, then gradually delete the old ones. Luckily the process is easy and straight forward as documented below.

Note: While I only have a single NSX-T Manager in my environment as it is a small home lab, usually in a production environment, you always want to maintain 3 NSX Managers active to sustain your NSX-T availability. Try to follow one of the below two approaches to maintain that.

  • Scenario A:
    • Manager A has IP address
    • Manager B has IP address
    • Manager C has IP address
    • Add Manager D with a new IP address, for example,
    • Remove Manager A.
    • Add Manager E with a new IP address, for example,
    • Remove Manager B.
    • Add Manager F with a new IP address, for example,
    • Remove Manager C.
  • Scenario B:
    • Manager A has IP address
Read More

Installation of NSX 6.4 VIB on ESXi 6.7 host failed

I have often got to interact with customers who had an issue getting the NSX VIB installed on their ESXi host. Most of the time, it is a tedious configuration issue or a step that they have forgotten. I have hit a similar issue today in my lab with me missing a simple step and wanted to share the error and the fix with others just in a hope it helps others recover from the same error quicker.

I was getting the following error every time I tried to install the NSX 6.4.5 VIB on my ESX 6.7U2 host, and similar error as well when I try to run the resolve button. The error stated “Unable to access agent VIB module at (_NSX_87_VTRES01_VMware Network Fabric). A screen shot of the errir is below.

Unable to access agent VIB Module at vxlan.zip

There was a more detailed error on my NSX screen, which unfortunately I seem to have lost the screenshot for, but it stated something like below:

vtesxi01.vt.com: Unable to access agent offline bundle at
Cause : <esxupdate-response>
<error errorClass=”MetadataDownloadError”>
<errorDesc>Failed to download metadata.</errorDesc>
<msg>(‘https://vtvc01.vt.com:443/eam/vib?id=ecf4a884-c9f5-406c-b57e-75a6613a3651’, ‘/tmp/tmpjnw369p9’, ‘[Errno 14] curl#6 – “Couldn\’t resolve host \’vtvc01.vt.com\'”‘)</msg>

As I have seen this one before, I was immediately able to spot that the fix is more than likely I have forgotten to setup Forward or Reverse DNS record or configuration for one of my setup component being ESXi, vCenter or NSX.… Read More

How to combat WannaCry Ransomware attack with VMware NSX

If you have not heard about the WannaCry Ransomware attacks lately, you need to get your head out of the sand for a bit longer. It sounds like this new Ransomware which take over the victom machine and encrypt their files and ask for a Bitcoin payment to give control back over their machine is taking the world by a storm. It is unbelievable how fast it has spread and how many machines it had took over in no time. It has even took over high profile organizations like Telefónica, Hitachi, Fedex,  National Health Service hospitals in England and Scotland, and many others.

ransomware wannacry compat with VMware NSX

While there has been  patches released by Microsoft to help mitigate the risk of this ransomware that uses a Windows vulnerability for Windows 7 and higher, there is currently no patches for earlier releases of Windows such as Windows XP.

While backup,  patching and keeping both your windows and security/antivirus software up to date is your first line of defense in such a situation, solutions such as VMware NSX can help you defend as well better contain such an attack. I have one of my colleagues “Angel Villar Garea (NSX Specialist SE in Spain)” has created a great video (see below), explaining how NSX can help to contain WannaCry in case a VM gets infected.… Read More

Integrating VMware NSX 6.3 with vRealize Automation 7.2

There is many reasons why you would want to integrated your vRA with NSX including on demand networking and security. If you have found this page, you probably had already figured out your reason, so I am not going to spend much time on that. Let’s get to how to do it!

In order to be able to use NSX on demand capabilities in your vRA, you will need to integrated the two together. Today this happen in vRA using the NSX plugin for vRO. The below instruction will document the steps you need to integrate your vRA 7.2 environment with VMware NSX 6.3. These instructions should work with a very little modification for other versions of vRA 7.x and NSX 6.x.

Install the NSX Plugin for vRealize Orchestrator

1- Download the NSX Plugin for vRO (Latest at the time of this post is 1.0.4) from the following link: NSX Plugin for vRO 1.0.4

2- Go to your vRO Control Center by going to: https://vtvro01.vt.com:8283/vco-controlcenter/ and login using root and the password you supplied during installation.

3- From Under Plugins click on the Manage Plug-Ins icon.

4- Hit Browse and choose the downloaded NSX for vRO Plugin (Mine was called: o11nplugin-nsx-1.0.4.vmoapp)… Read More

My VMworld 2017 VVD – NFV – SDN session

It’s that time of the year, where it is your chance to vote for the sessions you would like to see at VMworld. This year, I have submitted six VMworld Sessions focusing on VMware Validated Design (VVD), Network Function Virtualization (NFV), Software Defined Networking (SDN). I hope you find few you will like and vote for.

For those of you who are ready to rate the sessions, here is the list. If you like to ready the summary of all my sessions in one place, the you can read below. If you rate at least 3 of my sessions (high or low), please leave a comment below for a chance to win a $50 Amazon gift card.

Two VCDXs Deep Dive into VVD Network Stack. [2269] (Breakout Session)
VMware Validated Design The Why? Who? What? Why? & How? [2258] (Breakout Session)
Addressing the Most Common VMware Validated Design Decisions Deviations [2232](Breakout Session)
VMware Validated Design Experts Panel [2062] (Panel Discussion)
Software Defined Networking (SDN) vs Network Function Virtualization (NFV) [2242] (Breakout Session)
Question the VVD Network Stack Decisions [2277] (Panel Discussion)

Below is a summary of my sessions and looking forward for your votes and support.

Two VCDXs Deep Dive into VVD Network Stack.

Read More

Migrating Nexus 1000v to vDS in vRA environment

As VMware Distributed Switch has come a long way since it was first introduced in VMware vSphere 4.0. In vSphere 6.0, the Distributed Switch has became as rich on features and functionality as the Nexus 1000v at much lower complexity. While Nexus 1000v requires you to install/maintain an extra appliance(s) and VIB to use it’s features, vSphere Distributed Switch comes out of the box loaded with functionalities and ready to use. Not to mention the delay in upgrade to newer versions of vSphere you can encounter till Nexus 1000v is testing is completed.

The latest vSphere Distributed Switch has not left much to be desired from the Nexus 1000v to justify the extra complexity involved with the Nexus 1000v. All this has driven many customers to start migrating from Nexus 1000v to vDS lately, especially customers who is considering VMware SDDC/Cloud Solutions.  I have been involved with few of these migrations lately, and here will  share the migration process at a high level for the benefit of others going through the same process.

Below are the Nexus 1000v to vDS Migration steps at a high level:

1. Backup the Environment

a. Backup up vRA, vCD and any other management/Cloud platform that is consuming the environment.  … Read More

Kemp Technologies VLM-5000 load balancer review

A nice part of being vExpert is that different vendors reach out to you to try their products and you’re able to get an NFR licenses on cool technologies in return. Recently, I had the opportunity to try Kemp Technologies’ Virtual LoadMaster (VLM) 5000 application load balancer.  I was originally skeptical as there are many generic load balancer vendors out there, but there is few things about KEMP’s VLM-5000 that caught and kept my attention:

 KEMP’s load balancers are delivered in many form factors (Virtual Appliance, Hardware Load Balancer, Bare metal install, & even in the Cloud). Further, they cover most hypervisors out there (VMware vSphere, MS Hyper-V, KVM, Xen, & even Oracle Virtual Box). For each of these hypervisors they actually have built a virtual appliance specific to and optimized for it. I almost cannot think of a scenario, where they cannot get you covered. It is fair to mention though, I have only been able to test them on VMware vSphere as it’s my hypervisor of choice and I rather use a ready to go Virtual Appliance whenever possible.

Kemp Technologies load balancer formats

– I like Kemp Technologies applications approach to positioning their load balancers. I have found on their website a step by step documents that covers how to do load balance many of the most popular enterprise applications.… Read More

How to use vCAC new NetworkProfileName Custom Properties

Before the release of vCloud Automation Center 6.1.1, it was common to combine the use of the two below custom properties to assign a particular virtual machine to a particular portgroup/Network Path and a particular network profile:

VirtualMachine.NetworkN.Name: This custom property is used to put the virtual machine network adapter N, on the portgroup name supplied as a value for this custom property.

VirtualMachine.NetworkN.ProfileName: This custom property is used to tell the virtual machine network adapter N to obtain an IP from the network profile named in the value of this custom property.

I have seen the combined use of these two custom properties many time in the past and they seemed to work properly before vCAC 6.1.0 (It might have stopped working a bit earlier than that but I did not notice it). On the other hand using each of these custom properties on its own still work properly in vCloud Automation Center 6.1 and beyond, combining both custom properties on the same blueprint seems to produce some odd behaviors and unexpected results after 6.1. To avoid having such a problem, its highly recommended to use the newly introduced VirtualMachine.NetworkN.NetworkProfileName custom property.

vCAC NetworkProfileName custom property

VirtualMachine.NetworkN.NetworkProfileName kinda combine both custom properties in a single property.… Read More

Restarting vCloud Network & Security Manager Web Service

Some time it happen that your vCloud Networking & Security Manager (vShield Manager) stop behaving & you want to restart it. One of the scenarios where you might need this if it stop synchronizing properly with vCenter or vCloud Director. The good news is most of these problems were resolved with the latest release of vCloud Network & Security Manager, but if it happen and you need to restart your vCNS services then below the commands that you can execute at the vCloud Network & Security Manager Console without the need to reboot the vCNS Appliance which can take a while. Below is how to do so in a step by step fashion:

1- Login to your vCNS Console (Default user name & password are as follow u:admin & password: default, if you want to change them you can follow my earlier post: Changing vCNS Console Password

2- Change to enable mode using the following command:

manager> enable

3- Change to Configure Terminal mode using the following command:
manager# configure terminal

4- Stop the web-manager service using the following command:
manager(config)# no web-manager

5- Execute the following command to restart the web-manager service
manager(config)# web-manager

I usually recommend stopping the web-manager service before shutting down vCloud Network & Security Manager if required for maintenance or so on.… Read More

vShield Manager is not synched with vCenter Server after you disconnect and reconnect the vShield Manager vNic

I had the problem where every now and then powering on vAPPs that utilize vShield fails and it reports that it failed because it could not create the required port group. This used to drive me crazy, especially the way I have found out to fix it was to reboot the vCloud Networking & Security Manager (vShield Manager) to sync it again with vCloud Director. This has end up being a known bug with vCloud Networking & Security 5.1.2 (I believe the same problem existed with 5.1.1 as well, but not sure of earlier vCNS versions). It seems that vCloud Network & Security 5.1.2 fails to synch back with vCenter after its vNic lost connectivity for any reason even after connectivity being restored. This has been pointed out in the release note of 5.1.1 as follow:

vShield Manager not reachable after network interface is disconnected and reconnected
vShield Manager is not synched with vCenter Server after you disconnect and reconnect the vShield Manager vNic.
Workaround: Reboot vShield Manager.

Release notes for vCNS 5.1.1 can be found at: https://www.vmware.com/support/vshield/doc/releasenotes_vshield_511.html

I know, I know that work around in the release note of having to restart vCloud Network & Security Manager each time you have that problem is not the greatest solution especially if your network is on the shaky side as my home lab.… Read More

VIB module for agent is not installed on host (vShield-VXLAN-service)

While delivering a vCloud engagement to one of our enterprise customers using the latest vCloud Director 5.1.2 and vCloud Networking & Security 5.1.2a, my VXLAN configuration where failing at the stage where its preparing the hosts. I followed the same steps I have used for other customers and in my lab with previous versions, which is as well confirmed by different colleague to work and posted on other blogs.

For reference, the installation steps published by Rawlinson at http://www.punchingclouds.com/2012/09/09/vcloud-director-5-1-vxlan-configuration/ are almost identical to the installation step I have followed, though I kept getting the following error while the vCloud Networking & Security Manager is trying to prepare my hosts for VXLAN by pushing the VXLAN Agents to them:  “VIB module for agent is not installed on host   (vShield-VXLAN-service)”. The below images demonstrate the error I was getting in my vSphere Client and in the vCloud Networking & Security web interface:

VXLAN Agent VIB failed to install error message in vCenter


VXLAN not ready error in vCNS

After fuzzing with the error for couple of hours and researching a solution, I have discovered that for some reason vCloud Networking & Security is failing to automate the installation of the VXLAN Agent VIB into my ESXi hosts. For that I decided to try to install the VXLAN Agent VIB manually into my ESXi hosts & test if that work out.… Read More

Changing vCloud Networking & Security Console Password

While Changing the vCloud Networking & Security or vShield Manager Web Interface password is well documented at: Hardening vCloud Networking and Security 5.1.x virtual appliances, changing the console password of vCloud Network & Security or vShield Manager does not seems to be as well documented. Actually I have read in few places that its not possible to change the console password and enable password for vCloud Networking & Security Manager & Appliances. While that is partially true, you can actually recreate the admin account with the desired password which give you a similar effect to changing the password of the vCNS console admin account. The below procedure shows how to achieve just that in a step by step fashion:

How to change vCloud Networking & Security Console Password:

1. Connect to the console of the vShield Manager

2. Log in as ‘admin’  using the default credentials (U: admin  P: default)

3. Switch to ‘enable’ mode

  manager: enable

4. Switch to configuration mode

manager# configure terminal

5.  Create a temporary user, let’s call it tmpadmin

manager(config)# user tmpadmin password plaintext Newpassw0rd1

6. Save the configuration.

manager(config)# write memory

7. Exit twice until you are logged out

8. Log in as the new tmpadmin to the CLI and switch to ‘enable’ mode.… Read More

vCloud Networking & Security 5.1.1 create dvPort Groups, but fails to create vmknic interfaces

While installing vCloud Director 5.1 in my home lab, I have faced an odd problem while configuring vCloud Networking and Security 5.1.1 for VXLANs. If you follow VMware Configuration guides for VXLAN or any of the many articles on configuring vCloud Director/vCloud Networking & Security 5.1.1 for VXLAN, it will always mention that as soon you complete the configuration vCloud Networking & Security 5.1.1 will automatically create a dvPort Group that has a name of the format  vxw-vmknicPg-dvs-xx-xx-xx-xx, as well a vmknic interface. Few samples of such instructions can be found at:




In my lab I was facing the odd case of the dvPort Group being created, but no vmknic interface what so ever being created. After investigating the situation & a bit of internal research I have discovered that this is due to vCloud Networking and Security 5.1.1 depending on VMware Update Manager to push the VIB to each host to configure it for VXLAN, where in some cases VUM has proved problematic pushing these or a flaky VUM installation could cause such a problem. The good news is that vCloud Networking & Security 5.1.2a has just been released and handle pushing these VIBs differently and does not depend on VUM to do it eliminating all the trouble You can get the new vCloud Networking & Security 5.1.2a… Read More

VXLAN Concept Simplified

While VXLAN seems to be the next revolutionary and world changing network technology out there, all the marketing hypes around it makes too confusing for the rest of us. When VXLANs first came out, I have decided to learn more about it. While there was tons of materials about it online, the more I read about it the more confused I was about this new magical networking technology. I have to admit while I know my ways around networking, I am still not anything near CCIE or in another word a networking whiz. I believe many more people out there who is not in the networking field are still confused about what is VXLANs and what problems it came to solve. In this post, I am trying to over simplify VXLANs for the rest of us to understand it. You can call it VXLAN for dummies if you want. I am not going to cover VXLANs in depth, but to touch on what its and where the idea of it came from.

Let’s start by a quick definition of VXLAN. Virtual Extensible LAN (VXLAN) is a proposed encapsulation protocol for running an overlay network on existing Layer 3 infrastructure. An overlay network is a virtual network that is built on top of existing network Layer 2 and Layer 3 technologies to support elastic compute architectures.… Read More