Remove an unknown Disk from a vSAN Cluster

A failed capacity disk was removed from a vSAN 7 cluster before it could be logically detached from the disk group. The result is a remaining unknown disk device in the disk group that cannot be removed in the vSphere Client.

In such cases, esxcli is sometimes the more powerful tool.

We need to connect by SSH to the affected vSAN host.

Collect Information

Let’s check all registred disk devices on the node.

esxcli vsan storage list

A detailed list of all cache and capacity devices of this host will be displayed.

Output of disk devices (shortened)

Among the 24 active disks was the unknown zombie device. The only remaining feature was the vSAN UUID. The UUID can be used to detach the device from the configuration.

Remains of a physically removed disk in the vSAN configuration.

Extraction

The UUID of the missing unknown device was “52b17786-183b-e85f-f7f3-4befb19f67b0”. Using this information, we can remove it from the configuration.

esxcli vsan storage remove --uuid 52b17786-183b-e85f-f7f3-4befb19f67b0

The process takes a few seconds. Checking again with the esxcli vsan storage list command showed that the device was removed.

vExpert 2023 – Subprogram Nominations

VMware annually grants the vExpert award to individuals who have made a special contribution to the VMware community. This can be either through publications, presentations, blogs, or work in the VMware User Group (VMUG). I am pleased to be part of the vExpert community for the seventh year in a row in 2023.

In addition to the common vExpert, there are subprograms for specialized application branches.

I applied for the three sub-programs vExpertPro, Application-Modernization and Multi-Cloud and was accepted in all three categories.

vExpertPro

The mission of the vExpert PRO program is to create a global network of vExperts willing to find new vExperts in their local communities, support them, and mentor them on their way to becoming vExperts.

For this purpose, vExpertPro exist in many regions of the world. I have been a member of this group since 2021 myself and have been confirmed for another year.

vExpert Multi-Cloud

The multi-cloud area covers large parts of the VMware Compute portfolio. The term cloud includes not only the public cloud, but also local data centers (private cloud) and combinations of both approaches (hybrid cloud). This includes numerous products such as vSphere, vSAN, VMware Cloud Foundation (VCF), Aria, VMware Cloud on AWS, Site Recovery Manager (SRM) or vCloud Director (VCD).

I submitted my first application for this relatively new vExpert path in 2023 and was accepted. Many thanks to the business unit for the decision.

vExpert Application Modernization

Application Modernization is all about Tanzu and Kubernetes, as well as the ecosystem around these technologies. The background was described in great detail by Keith Lee in his article “Announcing the VMware Application Modernization vExpert Program 2023“.

Update Tanzu Workload Management

This is a brief guide on how to upgrade Tanzu Workload Management within the vSphere cluster.

Kubernetes Release and Patch Cycles

Kubernetes versions are specified as x.y.z following Semantic Versioning terminology, where x is the major version, y is the minor version, and z is the patch version. For example, v1.22.6 denotes a minor version 22 with patch level 6. Minor versions are released approximately every 3-4 months. In the meantime, there are several patches within the minor version.

The Kubernetes project maintains release branches for the last three minor versions (1.24, 1.23, 1.22). Since Kubernetes 1.19, newer versions receive patch support for about a year. So keeping the Kubernetes versions in Tanzu up to date is highly recommended.

Step 1 – Update vCenter

This step is not mandatory, but recommended. Updates on vCenter are often accompanied by a new Kubernetes versions. You can see notifications about updates in the vSphere Client.

Continue reading “Update Tanzu Workload Management”

Upgrade of a K3s Lightweight Kubernetes Cluster

K3s is a lightweight, highly available open source Kubernetes cluster platform designed for easy and resource-efficient installation. K3s is provided in a package of less than 60 MB. The package is optimized for ARM platforms and can therefore also be run on hardware such as a Raspberry Pi, or as a guest VM on ESXi-on-ARM.

Prerequisites and collection of information

K3s is a cluster solution. That is why the order in which the nodes are updated is important. The update starts on the master node. So first we need to find out which node has this role. The easiest way to do this is with a kubectl command:

kubectl get node
NAME STATUS ROLES AGE VERSION
k3node1.lab.local Ready master 2y43d v1.19.3+k3s3
k3node2.lab.local Ready none 2y42d v1.19.3+k3s3
k3node3.lab.local Ready none 2y42d v1.19.3+k3s3

From the output above we see my three K3s nodes with FQDN, status, role, age and version. So here k3node1 has the master role.

As an alternative, you can also execute the command in verbose mode:

kubectl get node -o wide
Continue reading “Upgrade of a K3s Lightweight Kubernetes Cluster”