VMware Cloud Director Cannot verify the Kubernetes API endpoint certificate on this Supervisor Cluster Error

While trying to Connect my VMware Cloud Director to my Tanzu Kubernetes Grid environment (TKGS) in vSphere 7 update 2, I kept hitting a certificate error. The error presented itself during the step to configure the Kubernetes policy for my Provider VDC. After following the wizard at:

Resources ==> Cloud Resources ==> Provider VDCs ==> Kubernetes.

The Wizard goes properly till I get to the Machine Classes tab which is where VMware Cloud Director get to check on the certificate of the supervisor cluster. That when the below error present itself:

“VMware Cloud Director cannot verify the Kubernetes API endpoint certificate on this Supervisor Cluster. It might be a vSphere generated default self-signed certificate or another invalid certificates. For the steps to install your own certificate, see Change the Kubernetes API Endpoint Certificate. For trusting the certificate, see https://kb.vmware.com/s/article/80996”

Kubernetes Policies in VCD 10.2 with vCenter 7.0 and later Tanzu are non-functional

The cause of the issue is the certificate structure of Tanzu Kubernetes in vCenter, The certificate of the Supervisor Cluster is not automatically trusted by VCD. Calls made to the Supervisor Cluster by VCD fail due to SSL errors. While I have this issue with VCD 10.2.2, it actually affected previous versions as well, especially when using self-signed certificates.

KB80996 article on how to fix this need to be updated with step marked in red below, but for those who needs to resolve the issue today, I wanted to document the steps in here.… Read More

Cloud VMware Cloud Flix Series for Cloud Providers

The More you Know, the Greater your Impact! The VMware Cloud Provider Team have created a series of 60-minute business and technical sessions that will enable you to capture business opportunities in a Multi-Cloud World. During these webinars, you will gain information and context to better understand the rapidly evolving Cloud Services market and how VMware can help you be successful in delivering compelling new services to your end customers.

To learn more please join our VMware CloudFlix series, see specific topics listed below. There are 10 insightful sessions coming up where our VMware experts will deep dive into the new technical Cloud capabilities.

VMware Cloud Flix educational sessions for VMware Cloud Providers

The initial list is below, I will keep adding to it as we add more sessions:

– May, 12th : Extending SDWAN services into your cloud platform with VeloCloud – Registration -> https://lnkd.in/eDmGp6Y

– May, 14th : How to move from NSX-V to NSX-T using the migration tool – Registration -> https://lnkd.in/ePJ6E6w

– May, 19th : VCD 10.1 What’s New – Registration -> https://lnkd.in/eB29mSM

– May, 26th: Cloud Director Service – Registration -> https://lnkd.in/eFXdTAJ

– May, 28th: CloudFlix Webinar – Container Service Overview for Service Providers​ – Registration -> http://tiny.cc/o23spz

– June, 2nd: vSAN reset – Registration -> http://tiny.cc/8chtozRead More

Cloud Director Kubernetes as a Service with CSE 2.6.x Demo

If you have been following our VMware Cloud Provider Space for a while, you have probably been introduced to our Cloud Director Kubernetes as a Service offering based on VMware Container Service Extension. In the past, Container Service Extension used to be command line only, where a nice UI was introduced in CSE 2.6.0. Here is a demo of what the new UI of Cloud Director Container Service Extension look like out of the box:

If you are curious of how to install CSE 2.6.1, you can follow my earlier post: VMware Container Service Extension 2.6.1 Installation step by stepRead More

VMware Container Service Extension 2.6.1 Installation step by step

One of the most requested feature with previous versions of the VMware Container Service Extension (CSE) is to add a native UI to it. As of CSE 2.6 we have added a native UI to CSE, which is adding to the friendliness of CSE and will make it much more appealing to many of our cloud providers. At just few clicks, our customers can deploy a K8S clusters at our Cloud Providers with filling few easy to understand fields.

Kubeconfig file will be auto generated as well and can be handed out right away to the developer limiting the efforts required by the tenant operation team/cloud providers administrators. Here is a quick screenshot teaser of what CSE look like. You can find a nice demo of CSE 2.6.1 UI at my following blog post: Cloud Director Kubernetes as a Service with CSE 2.6.x Demo

For more info on what is new with CSE 2.6.x please check my following blog post: vCloud Director Container Service Extension 2.6.x is here


Container Service Extension Create New Cluster


In this post, I am assuming you have an existing vCloud Director environment and AMQP already configured. To start the installation of CSE 2.6.1, you will need a supported OS. In my case, I have decided to go with CentOS 8.1.Read More

Running vCD Cli fail with the following error: ModuleNotFoundError: No module named ‘_sqlite3’

After installing the VMware Container Service Extension, which install the vCD CLI in the process, vCD CLI kept failing to start and complaining about not finding sqlite3 module as showing below.  I was installing on CentOS 8.1, but even then it sounds like the sqlite version included with CentOS is out of date for what vCD CLI require to be installed. Below is what the error looked like.

# vcd --help
Traceback (most recent call last):
  File "/home/rohan/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 18, in <module>
    from pysqlite2 import dbapi2 as sqlite3
ModuleNotFoundError: No module named 'pysqlite2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/rohan/.local/bin/vcd", line 6, in <module>
    from vcd_cli.vcd import vcd
  File "/home/rohan/.local/lib/python3.7/site-packages/vcd_cli/vcd.py", line 121, in <module>
    from vcd_cli import login  # NOQA
  File "/home/rohan/.local/lib/python3.7/site-packages/vcd_cli/login.py", line 24, in <module>
    from vcd_cli import browsercookie
  File "/home/rohan/.local/lib/python3.7/site-packages/vcd_cli/browsercookie/__init__.py", line 20, in <module>
    import sqlite3
  File "/usr/local/lib/python3.7/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/local/lib/python3.7/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'

While the steps below is mainly focused on how to fix the issue to be able to install vCD or CSE, the same steps will be relevant to anyone having a similar ModuleNotFoundError: No module named ‘_sqlite3’ error in Python to install any other application.… Read More

CSE 2.6.1 Error: Default template my_template with revision 0 not found. Unable to start CSE server.

While trying to run my Container Service Extension 2.6.1 after a successful installation. I kept getting the following error when trying to run CSE “Default template my_template with revision 0 not found. Unable to start CSE server.”

To fix this you will need to:

  1. Edit your CSE config.yaml file to include the right name of the default template and revision number. (Much more on this below)
  2. Encrypt your CSE config file again
  3. Re-Run CSE with your encrypted config file

In this post I will explain this in a bit more details for those hitting the same issue, as the resolution is quite simple but might not be as obvious if you are doing this for the first time.

For a start, here is what the exact error look like:

[root@vtcse01 Python-3.7.3]# cse run --config encrypted-config.yaml
Required Python version: = 3.7.3
Installed Python version: 3.7.3 (default, May  4 2020, 15:36:31)
[GCC 8.3.1 20190507 (Red Hat 8.3.1-4)]
Password for config file decryption:
Decrypting 'encrypted-config.yaml'
Validating config file 'encrypted-config.yaml'
Connected to AMQP server (VTAMQP01.vt.com:5672)
InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
Connected to vCloud Director (vcd.vt.com:443)
Connected to vCenter Server 'vtvc01' as 'administrator@vsphere.local' (vtvc01.vt.com:None)
Config file 'encrypted-config.yaml' is valid
Loading k8s template definition from catalog
Found K8 template 'ubuntu-16.04_k8-1.15_weave-2.5.2'
Read More

vCloud Director Container Service Extension 2.6.x is here!

As more and more of our Cloud Providers are being asked to support providing K8s and Container services to their customers in a self-service and Multi-tenant fashion, we have released Container Service Extension over 30 months ago.

The goal of Container Service Extension was to offer an Open Source plugin for vCloud Director that gives our Cloud Providers the capabilities to spin and scale Kubernetes Cluster with ease and minimum knowledge of Kubernetes by the Infrastructure team. It is an easy service that our Cloud Providers can add to their catalog at no extra cost specific to CSE. For more information about CSE, I would suggest you take a look at: https://vmware.github.io/container-service-extension/INTRO.html

VMware Container Service Extension

Our Container Service Extension had been evolving quickly and we just released CSE 2.6 . CSE 2.6 beta has been in testing for sometime by several partners. In this release, we had a lot of great enhancements coming including a native UI.

Here is a summary of features included with Container Service Extension 2.6:

  • New Templates with updated Kubernetes and Weave
  • In place Kubernetes upgrade for clusters
    • CSE offers the new capability to do in place upgrade of Kubernetes related software in Native clusters.
Read More