Skip to content

Field Dispatch: The First ESX Host Drifts from Custom Certificates After VMware Cloud Foundation 9 Instance Bring-Up

Effected Versions

  • VMware Cloud Foundation 9.0.x.x

If you're using securitySpec to deploy VMware Cloud Foundation 9.x with external CA-signed ESX certificates, the bring-up can look successful while still leaving you with one surprise:

Post Deployment Configuration Drift

The first ESX host will be deployed with a VMCA-signed certificate instead of the external CA-signed certificate provided.

I ran into this while testing external CA-signed certificates through both the API and Ansible. The JSON payload was accepted, the instance bring-up completed, the ESX hosts retained their external CA-signed certificates except for first ESX host, the one used during the bootstrap for the vCenter appliance.

What Happens

The behavior is straightforward if you understand the deployment process:

  1. You pass a securitySpec object in the deployment JSON.

    Example securitySpec in JSON Payload
    "securitySpec": {
     "esxiCertsMode": "Custom",
      "rootCaCerts": [
       {
         "alias": "custom-ca",
          "certChain": [
            "-----BEGIN CERTIFICATE-----\nMIIDqzC...\n...\n...==\n-----END CERTIFICATE-----"
         ]
       }
     ]
    }
    
  2. After the instance bring-up succeeds:

    1. The ESX certificate mode on management domain vCenter instance is set to Custom.
    2. The first ESX host is issued a VMCA-signed certificate.
    3. The remaining ESX hosts retain their external CA-signed certificates.

Why the Bootstrap ESX Host Is Different

This happens because the first ESX host is added to vCenter as part of the VCSA Installer bootstrap sequence, before the vCenter certificate management mode is switched to Custom.

The ESX host is discovered and managed while vCenter is still operating in its default certificate behavior. VMCA. Once the certificate mode is updated later in the bring-up process to Custom, the remaining hosts are added whilst retaining their pre-deployment custom certificate.

I haven't seen this present as a hard deployment failure. It presents as post-deployment certificate drift.

That configuration is sufficient to complete instance bring-up with external CA-signed certificates. The important nuance is that successful instance bring-up doesn't guarantee the bootstrap host kept the intended certificate.

What to Check After Instance Bring-Up

After deployment, inspect each ESX host certificate in the default cluster of the management domain.

If this behavior occurs in your environment, you should expect:

  1. Bootstrap ESX host: VMCA-signed certificate
  2. Remaining ESX hosts: External CA-signed certificates

Workaround

You will need to replace the certificate for the first ESX host again after instance bring-up.

In other words, treat the bootstrap host as a post-deployment cleanup item when using this method. The external CA-signed certificate can still be applied, but you may need to do it a second time after the VCF instance is fully deployed and vCenter is operating in the intended certificate mode.

That is not ideal, but it is operationally manageable if you know to expect it.

Why This Matters

This is exactly the kind of case that can slip past a deployment checklist. If the API accepts the payload and the installer completes, it is natural to assume the certificate state is correct everywhere. In this case, the deployment can still finish cleanly while leaving the bootstrap host out of alignment with the rest of the cluster.

For environments with strict certificate governance, that matters. One host drifting back to VMCA can create unnecessary audit noise, operational confusion, and extra remediation work after the fact.

The field takeaway is simple:

  1. Use securitySpec for external CA-signed ESX certificates if that fits your deployment pattern.
  2. Validate certificate state on every ESX host after instance bring-up.
  3. Expect the bootstrap host to be the exception.
  4. Replace certificate again on the first ESX host after instance bring-up.

Disclaimer

This is not an official VMware by Broadcom document. This is a personal blog post.

The information is provided as-is with no warranties and confers no rights.

Please, refer to official documentation for the most up-to-date information.