🚨 OpenStack Error Guide: 'Cluster status CREATE_FAILED' — Fix Magnum Kubernetes Cluster Provisioning

Overview

Cluster status CREATE_FAILED is the state Magnum reports when a Kubernetes (or other COE) cluster fails to provision. Magnum orchestrates the cluster by driving a Heat stack that boots the master/worker instances and runs cloud-init to configure Kubernetes, so a CREATE_FAILED cluster almost always maps to a CREATE_FAILED Heat stack — and the real error lives in the failed Heat resource, not the terse Magnum status.

The literal errors you will see:

openstack coe cluster show k8s-prod
| status              | CREATE_FAILED                                              |
| status_reason       | Stack CREATE failed: Resource CREATE failed: ...          |

ResourceInError: resources.kube_masters.resources[0].resources.kube-master: Went to status ERROR due to "Message: No valid host was found"

It occurs during openstack coe cluster create. The tell is that Magnum’s status is a summary; the actionable detail is one layer down in Heat, and often one more layer down in Nova/Neutron/cloud-init.

Symptoms

• openstack coe cluster list shows CREATE_FAILED.
• The cluster’s Heat stack is CREATE_FAILED.
• Master/worker instances are missing, in ERROR, or up but never join the cluster.

openstack coe cluster show <cluster> -c status -c status_reason -c stack_id

| status        | CREATE_FAILED                                       |
| status_reason | Stack CREATE failed: Resource CREATE failed: ...    |

docker logs magnum_conductor 2>&1 | grep -iE "CREATE_FAILED|ResourceInError|error" | tail -5

ERROR magnum.drivers.heat.driver ... Stack CREATE failed

Common Root Causes

1. Nova can’t schedule the cluster instances

If the flavor or capacity isn’t available, the Heat-driven server resource fails with No Valid Host.

openstack stack resource list <stack_id> -n5 --filter status=CREATE_FAILED
openstack stack resource show <stack_id> kube_masters 2>/dev/null | grep -i resource_status_reason

resource_status_reason | No valid host was found

See the dedicated No Valid Host guide for scheduling triage.

2. Wrong or missing cluster image / label mismatch

Magnum needs a specific Fedora CoreOS/Atomic-style image with the os_distro property; a wrong image or mismatched --label breaks cloud-init.

openstack coe cluster template show <template> -c image_id -c labels
openstack image show <image_id> -c properties | grep -i os_distro

os_distro='fedora-coreos'

If the image lacks the expected os_distro or the template labels reference an unsupported version, the nodes never configure Kubernetes.

3. Floating IP / network exhaustion

Cluster templates often assign floating IPs to masters; an exhausted pool or missing external network fails the stack.

openstack floating ip list -c "Floating IP Address" -c Status | tail
openstack coe cluster template show <template> -c external_network_id -c floating_ip_enabled

floating_ip_enabled | True
# but the external pool has no free addresses

4. cloud-init / Kubernetes bootstrap timeout

Instances boot but the software deployment (kubelet, etcd, control plane) times out — often no route to pull container images or to Keystone/Magnum for the cluster cert.

openstack stack resource list <stack_id> -n5 --filter status=CREATE_FAILED | grep -i deployment
openstack server list --name <cluster>- -c Name -c Status

| kube-master-0 | ACTIVE |  # up, but deployment resource timed out

An ACTIVE instance with a failed software-deployment resource points at bootstrap/networking, not scheduling.

5. Missing trust / service auth for the cluster

Magnum creates a trust so the cluster can call OpenStack (for cloud-provider, LBs, volumes); a broken trust or Keystone config fails bootstrap.

docker exec magnum_conductor grep -E 'trustee_domain|cluster_user_trust' /etc/magnum/magnum.conf
docker logs magnum_conductor 2>&1 | grep -iE "trust|trustee|unauthorized" | tail -5

trust_id ... could not be created / trustee_domain_id missing

6. No route to the container registry

If worker nodes can’t reach the registry to pull Kubernetes images, the deployment hangs then fails.

openstack coe cluster template show <template> -c labels | grep -iE 'container_infra_prefix|insecure_registry'

container_infra_prefix='registry.example.com/magnum/'

Diagnostic Workflow

Step 1: Get the real reason from Magnum, then Heat

openstack coe cluster show <cluster> -c status_reason -c stack_id
openstack stack show <stack_id> -c stack_status -c stack_status_reason

Step 2: Find the failed Heat resource

openstack stack resource list <stack_id> -n10 --filter status=CREATE_FAILED
openstack stack resource show <stack_id> <failed-resource> | grep -i resource_status_reason

This is the single most useful step — it turns “CREATE_FAILED” into a concrete cause.

Step 3: If it’s a server, triage Nova

openstack server list --name <cluster>- -c Name -c Status
openstack server show <instance> -c fault

No valid host → scheduling; ERROR spawn → image/network.

Step 4: If it’s a deployment, check bootstrap

openstack server list --name <cluster>- -c Name -c Status  # ACTIVE but deployment failed?
# console log for cloud-init errors
openstack console log show <master-instance> | tail -40

Step 5: Verify image, template, and trust config

openstack coe cluster template show <template> -c image_id -c external_network_id -c labels
docker exec magnum_conductor grep -E 'trustee_domain|region_name' /etc/magnum/magnum.conf

Example Root Cause Analysis

A user’s openstack coe cluster create leaves the cluster in CREATE_FAILED. Magnum’s reason is generic, so drill into Heat:

openstack coe cluster show k8s-prod -c stack_id
openstack stack resource list <stack_id> -n10 --filter status=CREATE_FAILED

| kube_masters | OS::Heat::ResourceGroup | CREATE_FAILED |

Drilling into the nested master resource:

openstack stack resource show <stack_id> kube_masters | grep -i resource_status_reason

resource_status_reason | ResourceInError: ... "No valid host was found. There are not enough hosts available."

The masters couldn’t schedule. The cluster template uses a large flavor, and Placement shows no host with enough free RAM after a busy week:

openstack coe cluster template show k8s-template -c flavor_id
openstack hypervisor stats show -c free_ram_mb -c free_vcpus

| free_ram_mb | 3072 |   # master flavor needs 8192

The fix is to free capacity (or use a smaller master flavor / different aggregate), then recreate:

# use a template with a schedulable flavor, or add capacity, then:
openstack coe cluster delete k8s-prod
openstack coe cluster create --cluster-template k8s-template-small k8s-prod
openstack coe cluster show k8s-prod -c status   # CREATE_IN_PROGRESS -> CREATE_COMPLETE

Longer term, size cluster-template flavors to available capacity and alert on hypervisor headroom so cluster creates don’t fail on scheduling.

Prevention Best Practices

• Always drill from the Magnum status into the failed Heat resource — openstack stack resource list --filter status=CREATE_FAILED is where the real cause lives.
• Validate the cluster-template image has the correct os_distro property and matches the labels/COE version before publishing the template.
• Ensure the floating-IP pool and external network have headroom for the masters (and workers if enabled) the template requests.
• Confirm nodes can reach the container registry and Keystone/Magnum endpoints; most ACTIVE-but-failed clusters are bootstrap/networking, not scheduling.
• Verify Magnum’s trust configuration (trustee_domain, region) so clusters can authenticate for cloud-provider integrations.
• Size cluster-template flavors to real hypervisor capacity and monitor headroom to avoid No Valid Host on create.
• Feed the failed Heat resource reason into the free incident assistant to classify scheduling vs bootstrap failures, and see more OpenStack guides.

Quick Command Reference

# Magnum -> Heat reason
openstack coe cluster show <cluster> -c status_reason -c stack_id
openstack stack show <stack_id> -c stack_status -c stack_status_reason

# The failed Heat resource (most useful step)
openstack stack resource list <stack_id> -n10 --filter status=CREATE_FAILED
openstack stack resource show <stack_id> <failed-resource> | grep -i resource_status_reason

# Server-side triage
openstack server list --name <cluster>- -c Name -c Status
openstack server show <instance> -c fault
openstack console log show <master-instance> | tail -40

# Template / image / trust
openstack coe cluster template show <template> -c image_id -c external_network_id -c labels
openstack image show <image_id> -c properties | grep -i os_distro
docker exec magnum_conductor grep -E 'trustee_domain|region_name' /etc/magnum/magnum.conf

# Recreate after fixing
openstack coe cluster delete <cluster>
openstack coe cluster create --cluster-template <template> <cluster>

Conclusion

Cluster status CREATE_FAILED is a Magnum summary of an underlying Heat stack failure; the actionable cause is in the failed Heat resource. Typical root causes:

1. Nova can’t schedule the cluster instances (No Valid Host).
2. A wrong cluster image or os_distro/label mismatch breaking cloud-init.
3. Floating-IP or external-network exhaustion.
4. A cloud-init/Kubernetes bootstrap timeout (often a registry/route problem).
5. A broken Magnum trust so the cluster can’t authenticate.
6. No route from nodes to the container registry.

Drill from the cluster status into stack resource list --filter status=CREATE_FAILED first — that single step converts an opaque CREATE_FAILED into a concrete Nova, Neutron, or bootstrap error you can actually fix.