Heads-up: Linux Start- and Reboot-Issues with Maxtang NX6412 solved

Cohesity vExpert gift

I recently became the owner of a Maxtang NX6412-B11 Mini PC. Cohesity gave away these barebones to vExperts at the VMware Explore EMEA in Barcelona. Once again a big thank you to Cohesity for their support of the community!

The fanless MiniPC with Elkhart Lake chipset is well-equipped. It has 2x 1 Gbit LAN, 1x USB-C (front), 2x USB 3.2 (front), 2x USB 2.0, 2x HDMI 2.0, and an audio jack.

Featured ports on the rear side.

The MiniPC will be a great addition to my homelab. I had intended to install the Tanzu community edition for it. Unfortunately, the project has since been discontinued by VMware and the removal of the packages from GitHub has been announced. 🙁

Hardware finish

The barebone still had to be provided with RAM and a flash disk. I installed a Samsung SSD 860 EVO Series 1TB M.2 SATA and two SO-DIMM DDR4 3200 16 GB from Crucial.

Reboot Issues with Linux

With the SATA SSD and the RAM, the machine was ready to boot. Ubuntu 22.04 LTS was used as operating system. After installation, a usual reboot was requested. However, the PC did not shut down completely and remained in the “Reached target shutdown” state. The PC had to be powered off hard. The reboot also took several minutes, which is very unusual for Ubuntu. To rule out the possibility that the problem is specific to Ubuntu, I tried an installation with Fedora. The result was exactly the same here too.

The solution

After a lengthy search, I found a clue that was specific to the EHL hardware platform. The fix is to disable a kernel module for the Intel Elkhart Lake SoC chipset. This can be done by adding it to the blacklist.conf file.

sudo vi /etc/modprobe.d/blacklist.conf

The line below must be added to blacklist.conf:

blacklist pinctrl_elkhartlake

Quit the vi editor with [ESC] [:] wq! (save and exit)

update-initramfs –u

The next shutdown was still delayed, but after a cold boot the OS came up within a few seconds.

I hope this hint helps someone – especially my vExpert colleagues who received the Cohesity gift too. Sharing is caring. 🙂

Running Tanzu Community Edition on a Linux VM – Simple Walkthrough for Beginners

You don’t need an enterprise cluster in order to get an impression of VMware Tanzu and Kubernetes. Thanks to the Tanzu Community Edition (TCE), now anyone can try it out for themselves – for free. The functionality offered is not limited in comparison to commercial Tanzu versions. The only thing you don’t get with TCE is professional support from VMware. Support is provided by the community via forums, Slack groups or Github. This is perfectly sufficient for a PoC cluster or the CKA exam training.

Deployment is pretty fast and after a couple of minutes you will have a functional Tanzu cluster.

TCE Architecture

The TCE can be deployed in two variants either as a standalone cluster or as a managed cluster.

Standalone Cluster

A fast and resource-efficient way of deployment without a management cluster. Ideal for small tests and demos. The standalone cluster offers no lifecycle management. Instead, it has a small footprint and can also be used on small environments.

Source: VMware

Managed Cluster

Like commercial Tanzu versions, there is a management cluster and 1 to n workload clusters. It comes with lifecycle management and cluster API. Thus, declarative configuration files can be used to define your Kubernetes cluster. For example, the number of nodes in the management cluster, the number of worker nodes, the version of the Ubuntu image or the Kubernetes version. Cluster API ensures compliance with the declaration. For example, if a worker node fails, it will be replaced automatically.

By using multiple nodes, the managed cluster of course also requires considerably more resources.

Source: VMware

Deployment options

TCE can be deployed either locally on a workstation by using Docker, in your own lab/datacenter on vSphere, or in the cloud on Azure or aws.

I have a licensed Tanzu with vSAN and NSX-T integration up and running in my lab. So TCE on vSphere would not really make sense here. Cloud resources on aws or Azure are expensive. Therefore, I would like to describe the smallest possible and most economical deployment of a standalone cluster using Docker. To do so, I will use a VM on VMware workstation. Alternatively, a VMware player or any other kind of hypervisor can be used.

Continue reading “Running Tanzu Community Edition on a Linux VM – Simple Walkthrough for Beginners”

Activate Immutability flag on existing Veeam XFS Repositories

Veeam Backup & Replication v11 is about to be launched on February 24th. Partners and service providers already have access to the RTM build.

One of my favourite features of version 11 are immutable backup files. You can read about it in my recent post “Hardening your Veeam Backup strategy with immutable repositories on Linux XFS“.

Some of you might already use Linux repository servers based on XFS with fast clone. Once you upgrade to Veeam Backup & Replication version 11 you may wish to activate immutable backups. I will explain in this blog post how you can use this new feature on existing XFS repository servers.

Activate immutability

To activate the immutability feature edit your existing backup repository servers and go to step “repository” (picture below). Starting with version 11 there’s a checkbox “make recent backups immutable for [x] days” where 7 is the minimum value to enter. Once you activate the checkbox you will most likely see a message “Unable to set the immutability option: cannot find the transport service on server <your repo server>. Configure the server anyway?“. Click on [Yes].

To make use of the immutability flag you’ll have to reconfigure the transport service on the Linux server.

Continue reading “Activate Immutability flag on existing Veeam XFS Repositories”

Hardening your Veeam Backup strategy with immutable repositories on Linux XFS

Attackers on IT infrastructure have become more sophisticated in recent times. Not only do they ecrypt live data and entire virtual machines, they also have learned to delete or encrypt entire backups too. No matter if your online backups are stored on a local repository or a cloud share. If that happens and you do not have an offline backup like tape or any other air gapped solution you’re up a creek without a paddle. Even worse: If you’re responsible for data backup, you might be facing unpleasant questions by the management.

Now the question is how to protect backup data in case an attacker gets hold on your backup server. Once they get credentials to the server, they can do whatever the backup-admin can do. Deleting backups for example. One first step is to separate your backup systems from your domain. In case the domain administrator account gets compromised there’s still (a little) barrier between attackers and your backup systems. But still, you’ll never know if attackers have knowledge of a zeroday exploit to takeover your backup server. It would be a better approach to protect backups against deletion for a defined timespan. There’s a similar option on AWS S3 buckets, which are usually used as offsite backup copies. Getting back offsite backups takes a considerable amount of time. In case of a crypto attack time is crucial and the clock is ticking. Wouldn’t it be great to have immutable backups on your primary repositories on site? Good news. There’s light at the end of the tunnel (and I promise it’s not the oncoming train).

Immutable Backups

Before we go into detail, we need to clarify what immutable backups mean. The idea is that you write once and then files (backups) are protected for a self defined period of time. Even the backup administrator cannot delete them before the defined timespan has passed.

Veeam Backup & Replication v11 will make use of a native Linux XFS filesystem feature. XFS can set an extended file attribute [i] which will protect the file from renaming, modification, deletion or hard-linking. The key point is that backups are transferred and written with non-privileged accounts and the immutable attribute is set and removed by a privileged service on the repository server which cannot be accessed from outside.

All you need is a Linux repository formatted with XFS and Veeam Backup & Replication v11, which will be released in the near future. (Update: Expected release date will be February 24th 2021)

Continue reading “Hardening your Veeam Backup strategy with immutable repositories on Linux XFS”