🚨 OpenStack Error Guide: 'internal error: process exited while connecting to monitor' — fix QEMU/libvirt spawn

Overview

internal error: process exited while connecting to monitor is the libvirt error Nova surfaces when nova-compute asks libvirt to boot an instance, libvirt launches the QEMU process, but that process dies almost immediately — before libvirt can attach to its QMP monitor socket. The instance goes to ERROR with Instance failed to spawn, and the real reason is in the QEMU stderr that libvirt captured, not in the generic Nova message.

The literal errors you will see:

libvirt.libvirtError: internal error: process exited while connecting to monitor: 2026-07-06T02:11:44.882Z qemu-system-x86_64: -device ... : Property '...' not found

nova.compute.manager [instance: ...] Instance failed to spawn: libvirt.libvirtError: internal error: process exited while connecting to monitor

It occurs at spawn time on a specific compute host: a nova boot, a live-migration target, or an evacuation. Because QEMU exits before the monitor is up, libvirt only knows “it died” — you have to read QEMU’s own last words to find out why.

Symptoms

• New instances go straight to ERROR on one or more compute hosts; nova show reports Instance failed to spawn.
• nova-compute.log (or docker logs nova_compute) shows the process exited while connecting to monitor libvirt error.
• Often host-specific: the same flavor/image boots fine on another compute node.

openstack server show <instance> -c fault -c status -c "OS-EXT-SRV-ATTR:host"

| status | ERROR |
| fault  | {'message': 'internal error: process exited while connecting to monitor...'} |
| OS-EXT-SRV-ATTR:host | compute-03 |

docker logs nova_compute 2>&1 | grep -iE "process exited|failed to spawn|libvirtError" | tail -3

ERROR nova.compute.manager Instance failed to spawn: libvirt.libvirtError: internal error: process exited while connecting to monitor

Common Root Causes

1. CPU model / machine type not supported by the host

A flavor or image asks for a CPU feature, model, or machine type the host QEMU/KVM can’t provide (common after moving a workload to older hardware or a mixed fleet). QEMU exits complaining about the CPU or a device.

docker exec nova_libvirt virsh capabilities 2>/dev/null | grep -iE "<model>|machine" | head
docker exec nova_compute grep -iE 'cpu_mode|cpu_model|hw_machine_type' /etc/nova/nova.conf

qemu-system-x86_64: warning: host doesn't support requested feature ...
qemu-system-x86_64: Some features were not available: ...

2. Hugepages / NUMA memory not available

If the flavor requests hugepage-backed memory (hw:mem_page_size) but the host has none free (or none configured), QEMU can’t allocate its RAM and exits.

grep -R HugePages /proc/meminfo
docker exec nova_libvirt virsh freepages --all 2>/dev/null

HugePages_Total:       0
HugePages_Free:        0

qemu-system-x86_64: unable to map backing store for guest RAM: Cannot allocate memory

3. Nested virtualization / KVM unavailable

If /dev/kvm is missing or the container can’t access it, QEMU falls back or fails on a KVM-specific device.

ls -l /dev/kvm
docker exec nova_libvirt virsh domcapabilities 2>/dev/null | grep -i kvm

ls: cannot access '/dev/kvm': No such file or directory

4. Corrupt or wrong-format image / disk

A truncated qcow2, a disk_format that doesn’t match the actual bytes, or a bad backing file makes QEMU exit while opening the drive.

INST=$(openstack server show <instance> -f value -c OS-EXT-SRV-ATTR:instance_name)
docker exec nova_libvirt qemu-img info /var/lib/nova/instances/<uuid>/disk 2>/dev/null

qemu-system-x86_64: -drive ...: Could not open backing file: Invalid argument

5. SELinux / AppArmor / svirt denial

A mislabeled disk or device path can be blocked by sVirt (SELinux) or AppArmor, so QEMU dies on an EACCES opening its own files.

sudo ausearch -m avc -ts recent 2>/dev/null | grep -i qemu | tail
sudo dmesg | grep -iE 'apparmor|denied' | tail

type=AVC ... denied { read } ... comm="qemu-system-x86" ...

6. Unsupported or malformed device flag

An image property or flavor extra-spec injects a device/option (vTPM, virtio feature, PCI address) the host QEMU build doesn’t have — QEMU prints “Property not found” and exits.

docker exec nova_libvirt virsh dumpxml <instance_name> 2>/dev/null | grep -iE 'tpm|model|address' | head
openstack image show <image> -f value -c properties

qemu-system-x86_64: -device tpm-crb...: Property 'tpm-crb.tpmdev' not found

Diagnostic Workflow

Step 1: Get the real QEMU error, not just Nova’s

The libvirt message truncates. Read the per-instance QEMU log where the full stderr lands.

INST=$(openstack server show <instance> -f value -c OS-EXT-SRV-ATTR:instance_name)
docker exec nova_libvirt tail -30 /var/log/kolla/libvirt/qemu/${INST}.log

The last QEMU line before it exits is the actual cause.

Step 2: Confirm which host and whether it’s host-specific

openstack server show <instance> -c "OS-EXT-SRV-ATTR:host" -c fault

If it boots on other hosts, focus on this host’s CPU/hugepages/KVM/labels.

Step 3: Check host virtualization capabilities

ls -l /dev/kvm
docker exec nova_libvirt virsh capabilities | grep -iE "<model>|machine" | head
docker exec nova_libvirt virsh domcapabilities | grep -iE 'kvm|machine|maxCpus'

Step 4: Check memory / hugepages if the flavor uses them

openstack flavor show <flavor> -f value -c properties   # look for hw:mem_page_size, hw:numa*
grep -R HugePages /proc/meminfo
docker exec nova_libvirt virsh freepages --all

Step 5: Validate the image / disk

openstack image show <image> -c disk_format -c properties
docker exec nova_libvirt qemu-img info /var/lib/nova/instances/<uuid>/disk

Step 6: Check for security-module denials

sudo ausearch -m avc -ts recent | grep -i qemu | tail
sudo dmesg | grep -iE 'apparmor|denied' | tail

Example Root Cause Analysis

A tenant reports that a new flavor fails to boot on compute-03 but works elsewhere. nova show gives the generic process exited while connecting to monitor. Reading the QEMU log directly:

INST=$(openstack server show demo-vm -f value -c OS-EXT-SRV-ATTR:instance_name)
docker exec nova_libvirt tail -5 /var/log/kolla/libvirt/qemu/${INST}.log

qemu-system-x86_64: unable to map backing store for guest RAM: Cannot allocate memory

That’s not a CPU or image problem — QEMU couldn’t get its RAM. The flavor requests 1 GiB hugepages:

openstack flavor show hp-large -f value -c properties

hw:mem_page_size='1GB'

But compute-03 has no free 1 GiB hugepages:

docker exec nova_libvirt virsh freepages --all | grep -A3 1048576

1048576KiB: 0

The host was rebooted and the hugepage allocation didn’t persist. Fix: restore the hugepage reservation (via kernel cmdline / sysfs and Kolla host config), confirm free pages, and retry:

echo 8 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
docker exec nova_libvirt virsh freepages --all | grep -A3 1048576   # now non-zero
openstack server create --flavor hp-large --image <img> --network <net> demo-vm2  # ACTIVE

Longer term, make the hugepage reservation persistent (kernel cmdline hugepagesz=1G hugepages=N) so a reboot can’t silently strip capacity from the compute host.

Prevention Best Practices

• Read the per-instance QEMU log (/var/log/kolla/libvirt/qemu/<name>.log) first — it holds the real exit reason the generic libvirt/Nova message hides.
• Keep compute hosts homogeneous, or pin cpu_mode/hw_machine_type and use host aggregates so a flavor never lands on a host that can’t provide its CPU/machine type.
• Make hugepage and NUMA reservations persistent (kernel cmdline) and monitor virsh freepages; schedule hugepage flavors only onto hosts that advertise the capacity.
• Verify /dev/kvm is present and passed into nova_libvirt on every compute host; alert if it disappears.
• Validate images at upload time (qemu-img info, correct disk_format) so a truncated or mislabeled image can’t reach spawn.
• Keep SELinux/AppArmor policies current with the Kolla images; investigate AVC denials rather than disabling enforcement.
• For triage, paste the QEMU log’s final lines into the free incident assistant to classify the exit reason, and see more OpenStack guides.

Quick Command Reference

# The REAL error (per-instance QEMU stderr)
INST=$(openstack server show <instance> -f value -c OS-EXT-SRV-ATTR:instance_name)
docker exec nova_libvirt tail -30 /var/log/kolla/libvirt/qemu/${INST}.log

# Which host, and Nova's view
openstack server show <instance> -c fault -c status -c "OS-EXT-SRV-ATTR:host"
docker logs nova_compute 2>&1 | grep -iE "process exited|failed to spawn" | tail -10

# Host virtualization capabilities
ls -l /dev/kvm
docker exec nova_libvirt virsh capabilities | grep -iE "<model>|machine" | head
docker exec nova_libvirt virsh domcapabilities | grep -iE 'kvm|machine'

# Hugepages / NUMA
openstack flavor show <flavor> -f value -c properties
grep -R HugePages /proc/meminfo
docker exec nova_libvirt virsh freepages --all

# Image / disk & security denials
docker exec nova_libvirt qemu-img info /var/lib/nova/instances/<uuid>/disk
sudo ausearch -m avc -ts recent | grep -i qemu | tail

Conclusion

internal error: process exited while connecting to monitor means QEMU died before libvirt could attach to its monitor — so the generic Nova/libvirt message is never the real reason. The fix always starts by reading the per-instance QEMU log for its final stderr line. Typical root causes:

1. A CPU model / machine type the host can’t provide.
2. Hugepage/NUMA memory the flavor needs but the host doesn’t have free.
3. Missing /dev/kvm or unavailable KVM acceleration.
4. A corrupt image or a disk_format that doesn’t match the bytes.
5. An SELinux/AppArmor (sVirt) denial blocking QEMU’s own files.
6. An unsupported device/flag (vTPM, virtio feature) QEMU rejects.

Confirm whether the failure is host-specific, then let the last QEMU log line point you at CPU, memory, KVM, image, labels, or device flags — that single line collapses the search space.