Terraform apply cpu/memory change would reboot guest VM even if hot add is enabled for CPU and memory

[sgaosdgr]

Community Guidelines

• I have read and agree to the HashiCorp Community Guidelines .
• Vote on this issue by adding a 👍 reaction to the original issue initial description to help the maintainers prioritize.
• Do not leave "+1" or other comments that do not add relevant information or questions.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform

v1.8.1

Terraform Provider

v2.7.0

VMware vSphere

v8.0 u2b

Description

When hot add is enabled for CPU and memory, increasing CPU count and memory size should take effect immediately without rebooting the target VM.

However, this is not the case. Increasing CPU and memory size both cause the target VM rebooting immediately.

The guest OS is RockyLinux 9.3 x86_64. Open-vm-tools 12.2.5 is installed.

Affected Resources or Data Sources

vm/

Terraform Configuration

#VMware vSphere Provider
provider "vsphere" {  
  #Set of variables used to connect to the vCenter
    vsphere_server = var.vsphere_server 
    user           = var.vsphere_user
    password       = var.vsphere_password
  
#If you have a self-signed cert
    allow_unverified_ssl = true
  }
  
#Name of the Datacenter in the vCenter
  data "vsphere_datacenter" "dc" {
    name = "DC1"
  }
#Name of the Cluster in the vCenter
  data "vsphere_compute_cluster" "cluster" {
    name          = "Cluster"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Datastore in the vCenter, where VM will be deployed
  data "vsphere_datastore" "datastore" {
    name          = "vSAN"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Portgroup in the vCenter, to which VM will be attached
  data "vsphere_network" "external_network" {
    name          = "External"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Portgroup in the vCenter, to which VM will be attached
  data "vsphere_network" "internal_network" {
    name          = "Public"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
#Name of the Templete in the vCenter, which will be used to the deployment
  data "vsphere_virtual_machine" "rocky9-template" {
    name          = "rocky9-temp"
    datacenter_id = data.vsphere_datacenter.dc.id
  }
  
#Set VM parameteres
  resource "vsphere_virtual_machine" "rocky9-vm" {
    name = "rocky91"
    num_cpus = 2
    memory   = 4096
    firmware = "efi"
    cpu_hot_add_enabled = "true"
    memory_hot_add_enabled = "true"
    guest_id = "rockylinux_64Guest"
    resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
    datastore_id     = data.vsphere_datastore.datastore.id

    network_interface {
      network_id = data.vsphere_network.external_network.id
    }
    
    network_interface {
      network_id = data.vsphere_network.internal_network.id
    }

    disk {
      label            = "disk0"
      thin_provisioned = true
      size             = 50
    }
  
    clone {
      template_uuid = data.vsphere_virtual_machine.rocky9-template.id
#Linux_options are required section, while deploying Linux virtual machines
      customize {
          linux_options {
              host_name = "rocky91"
              domain = "net.local"
          }
          network_interface {
            ipv4_address = "10.10.1.5"
            ipv4_netmask = "23"
          }
          network_interface {
            #ipv4_address = "192.168.80.11"
            #ipv4_netmask = "24"
          }
          ipv4_gateway = "10.10.1.1"
      }
      
    }

  }

#Outup section will display vsphere_virtual_machine.ubu-testing Name and IP Address
output "VM_Name" {
  value = vsphere_virtual_machine.rocky9-vm.name
}

output "VM_IP_Address" {
  value = vsphere_virtual_machine.rocky9-vm.guest_ip_addresses
}

Debug Output

The guest VM got rebooted immediately during terraform apply

Panic Output

No response

Expected Behavior

When increasing CPU core or increasing memory size, the change takes effect immediately without rebooting the guest VM.

Actual Behavior

Both increasing CPU and memory will cause the guest VM rebooting immediately.

Steps to Reproduce

1. Create a template for RockyLinux 9 VM
2. Install open-vm-tools in the template
3. Provisioning an VM with the terraform code listed above
4. Try adding CPU core or increasing memory size

Environment Details

No response

Screenshots

No response

References

No response

[github-actions]

Hello, sgaosdgr! 🖐

Thank you for submitting an issue for this provider. The issue will now enter into the issue lifecycle.

If you want to contribute to this project, please review the contributing guidelines and information on submitting pull requests.

[sgaosdgr]

Some updates:

The reboot is caused by two fields:

ept_rvi_mode = null
hv_mode = null

At each apply, they gets reset even if these fields are already set to on/hvOn, but the field values are not registered/recorded.

[sgaosdgr]

Actually no matter what values are set, automatic, on/hvOn, off/hvOff, these fields always have a null value. This is the very reason at each play the target VM will be rebooted.

  # vsphere_virtual_machine.rocky9-vm will be updated in-place
  ~ resource "vsphere_virtual_machine" "rocky9-vm" {
      + ept_rvi_mode                            = "off"
      + hv_mode                                 = "hvOff"

[sgaosdgr]

Just more information: experienced the same issue with Terraform 1.7.5. For debugging, we upgraded Terraform from 1.7.5 to 1.8.1. But terraform upgrade does not make any difference.

[sgaosdgr]

This is a serious issue. This bug can reboot all VMs managed by terraform/vsphere at once whether an VM has any change or not.

[tenthirtyam]

Looks like those are not getting computed.

[sgaosdgr]

Actually found the similar issue reported in #1902.

After applying the workaround, it does not reboot every VMs (with no change to them) any more.

However, changing CPU and memory size still trigger VM reboot. Manually adjusting CPU and memory does not cause VM reboot.

[sgaosdgr]

For example, memory change:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # vsphere_virtual_machine.default["vm1"] will be updated in-place
  ~ resource "vsphere_virtual_machine" "default" {
        id                                      = "421ae1e6-f4cf-6ab7-2ea7-3ec005a97055"
      ~ memory                                  = 4096 -> 6144
        name                                    = "vm1"
        tags                                    = []
        # (72 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

This change reboots the VM.

[tenthirtyam]

Duplicate of #1902

[sgaosdgr]

It's not a duplicate. It's about hot add CPU and memory. Hot add CPU/memory support is broken right now.

[sgaosdgr]

terraform show:

    memory                                  = 6144
    memory_hot_add_enabled                  = true

   cpu_hot_add_enabled                     = true

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.