Update error VCSA 7 – vCenter Server not operational

During patching of a vCenter server appliance (VCSA) problems can occur. Maybe contact to the update source was lost or the whole process has been cancelled by an operator. If you try to reapply the patch, you might see an error like in the picture below.

Update Installation failed. VCenter Server is not operational.

In the VAMI interface of vCenter everything looks fine. All services are up and running and ovarall status is green. Even a reboot of the appliance doesn’t help. The source of the problem lies in an interrupted update procedure which leaves a status file behind. We need to fix (remove) that manually.

To do so open a SSH shell to the vCenter server appliance and change to the directory where the file was left.

# cd /etc/applmgmt/appliance

You’ll see a file called software_update_state.conf. Under normal circumstances this file will be removed after an update. But something went wrong and it wasn’t cleaned up. Let’s have a brief look inside the file.

# cat software_update_state.conf
{
"state": "INSTALL_FAILED",
"version": "7.0.0.10700",
"latest_query_time": "2020-09-17T11:42:37Z"
}

We can see that there’s been a failed update to VCSA 7.0.0.10700. You can just remove the file.

# rm software_update_state.conf

If you now trigger a new patch installation it will work.

Upgrade vCenter Server Appliance to vSphere 7

Recently I’ve upgraded my homelab from 6.7U3 to vSphere7. The workflow is straightforward and very easy. The VMware Design team did a very good job with the UI.

First steps

I cannot point that out enough: check the VMware HCL. Just because your system is supported under your current vSphere version, doesn’t mean it’ll be supported under vSphere7 too. On the day I’ve upgraded, vSphere7 was brand new and there were just a few entries in the HCL. But it’s a homelab and if something breaks I don’t care to rebuild it from scratch. Don’t do this in production!

Although my Supermicro E300-9D is not yet certified for version 7.0, it works like a charm. I guess it’s just a matter of time, because the VMware Nano-Edge cluster is based on that hardware.

Before we can start, you need to download the vCenter Server Appliance 7.0 (VCSA) from VMware downloads (Login required). You also need to have new license keys for vCenter, ESXi and vSAN (if yor cluster is hyperconverged).

Continue reading “Upgrade vCenter Server Appliance to vSphere 7”

Runecast 3.0 requires elevated privileges for HCL checks

A couple of days ago Runecast Analyzer has been upgraded to version 3.0.0. With that upgrade a very important beta-feature became GA: HW Compatibility and Upgrade Simulator.

I used to run the Runecast service account with readonly privileges. It has been sufficient up to version 2.7.x. Even the hardware compatibility check (beta) did work with readonly privileges. After upgrading my appliance to version 3.0.0 (GA), I found a notification. Missing privileges..

Once you open host details and click on I/O devices tab, there’s further information.

Continue reading “Runecast 3.0 requires elevated privileges for HCL checks”

VCSA 6.7 archive partition filling up

During a cluster routine check with RVtools I’ve seen a warning.

Warnings like this on a vCenter appliance let my alarm bells ring. Partitions filling up close to 100% are not a desirable condition. But let’s have a look into VAMI (vCenter Appliance Management Interface).

https://<vcsa_FQDN>:5480

Now that’s strange. Everything seems to be in a green state. It’s a VCSA 6.7 Update 2a (6.7.0.31000). Older versions did show a storage warning. “File system /storage/archive is low on storage space“. But this appliance seems to be happy. Let’s have a look at the shell, to see what’s really happening.

It looks like mount point /storage/archive is almost filled up.

Continue reading “VCSA 6.7 archive partition filling up”