🚨 OpenStack Error Guide: 'Placement 409 Conflict' allocation update failure

Exact Error Message

2026-06-27 17:14:55.402 7 WARNING nova.scheduler.client.report
[req-2c1a9d44-7b10-4e02-9a31-8c2b1d3e4400 - - - - -] Unable to submit allocation for instance
b3d2... (409 {"errors": [{"status": 409, "title": "Conflict",
"detail": "There was a conflict when trying to complete your request.\n\n update conflict:
Inventory for VCPU on resource provider 8a1c... in use.", "request_id": "req-..."}]})

Another common form on inventory updates:

409 Conflict: resource provider generation conflict; expected 42 but got 41

What the Error Means

The Placement service uses optimistic concurrency control. Every resource provider and every consumer’s allocations carry a generation number that increments on each successful write. When a client (Nova’s scheduler.client.report, or nova-manage) sends an update, it includes the generation it last saw. If the stored generation has moved on since — because another writer updated that provider or consumer first — Placement rejects the write with 409 Conflict rather than silently overwriting.

So a 409 is not corruption; it is Placement protecting against lost updates. Most 409s are transient: two writers raced, the loser retries with the fresh generation, and it succeeds. A persistent 409, however, signals a real problem — a stuck generation mismatch, an inventory that cannot be reduced because existing allocations exceed the new value, or stale/duplicate resource providers left by a removed or renamed compute host.

Common Causes

• Concurrent allocation writes — two operations updated the same provider or consumer simultaneously; the loser sees a stale generation.
• Inventory reduction below current usage — update conflict: Inventory for VCPU ... in use when you try to lower capacity below what is already allocated.
• Stale resource provider after host rename/removal — the compute node re-registered under a new UUID, leaving an old provider that conflicts.
• Allocation drift — Nova’s view of allocations diverged from Placement, so heal attempts collide.
• Aggregate / trait writes racing — concurrent nova-manage placement operations on the same provider.
• Clock or retry storms — a flapping compute service repeatedly rewriting inventory, generating a stream of 409s.

How to Reproduce the Error

The inventory-in-use variant is deterministic:

1. Identify a compute resource provider with active VCPU allocations.
2. Attempt to set its VCPU inventory total below the amount currently allocated.

openstack resource provider inventory list <rp-uuid>
# try to shrink VCPU below the allocated sum
openstack resource provider inventory set <rp-uuid> --resource VCPU=2:2

Placement rejects the reduction with 409 Conflict: Inventory for VCPU ... in use. The generation-mismatch variant appears naturally under concurrent scheduler writes to a busy provider.

Diagnostic Commands

Read-only. Establish whether the 409 is transient contention or a genuine inventory/provider conflict.

# List providers and their current generation
openstack resource provider list
openstack resource provider show <rp-uuid> -c uuid -c generation -c name

# Current inventory vs current usage on the provider
openstack resource provider inventory list <rp-uuid>
openstack resource provider usage show <rp-uuid>

+----------------+------------------+----------+----------+----------+-------+
| resource_class | allocation_ratio | total    | reserved | min_unit | used  |
+----------------+------------------+----------+----------+----------+-------+
| VCPU           |             16.0 | 64       | 0        | 1        | 58    |
+----------------+------------------+----------+----------+----------+-------+

Check for duplicate/stale providers and read the Nova/Placement logs:

# Two providers for the same hostname is a red flag
openstack resource provider list --name <compute-hostname>

# Kolla-Ansible
docker logs placement_api 2>&1 | grep -i "409\|conflict" | tail
docker logs nova_scheduler 2>&1 | grep -i "409\|generation conflict" | tail
# Traditional packages
journalctl -u openstack-placement-api | grep -i "409\|conflict" | tail
journalctl -u openstack-nova-scheduler | grep -i "409\|generation conflict" | tail

Step-by-Step Resolution

1. Decide whether the 409 is transient or persistent. A few 409s per hour during heavy scheduling are normal — the client retries and succeeds. Only a repeating failure on the same provider/consumer needs intervention.

2. For Inventory ... in use, compare inventory to usage. If you are trying to shrink inventory below used, that is the conflict. Either reduce usage first (migrate/delete instances) or set the inventory no lower than current usage:

   openstack resource provider usage show <rp-uuid>

3. Hunt for duplicate resource providers. If a host re-registered under a new UUID, an old provider lingers:

   openstack resource provider list --name <compute-hostname>

   The stale provider (with no live compute service) is the source of conflicting writes.

4. Heal allocation drift when Nova and Placement disagree about consumer allocations:

   nova-manage placement heal_allocations --instance <instance-uuid>

   This reconciles the consumer’s allocations to a consistent generation.

5. Restart the flapping writer if a compute service is rewriting inventory in a loop and generating a 409 storm:

   docker restart nova_compute            # Kolla-Ansible
   systemctl restart openstack-nova-compute   # Traditional

6. Re-verify the provider’s generation advances cleanly and scheduling succeeds:

   openstack resource provider show <rp-uuid> -c generation

Prevention and Best Practices

• Treat occasional 409s as normal optimistic-concurrency retries — alert only on sustained conflicts against the same provider or consumer.
• Never shrink a provider’s inventory below its current usage; migrate or delete the consuming instances first.
• Clean up stale resource providers promptly after renaming or decommissioning a compute host so no orphan provider competes for writes.
• Run nova-manage placement heal_allocations on a schedule (or after any control-plane incident) to keep Nova and Placement allocations in sync.
• Avoid running multiple nova-manage placement inventory/aggregate operations against the same provider concurrently.

Related Errors

• 409 Conflict: resource provider generation conflict; expected N but got M — the pure optimistic-concurrency variant, almost always transient.
• 409 Conflict: Inventory for <class> on resource provider ... in use — an inventory reduction below current usage.
• Resource provider not found / No allocation candidates — adjacent Placement failures covered standalone in our /categories/openstack/ guides.
• Unable to submit allocation for instance warnings in nova-scheduler — the Nova-side symptom of repeated 409s.

Frequently Asked Questions

Is a 409 Conflict an error I must act on? Not always. Placement uses optimistic concurrency; a 409 means another writer got there first and the client should retry with the fresh generation. Only persistent 409s on the same target indicate a real problem.

Why can’t I lower a provider’s VCPU inventory? Because instances are currently allocated against it. Placement refuses to set inventory below used to avoid over-allocating. Reduce usage first, or set inventory at or above current usage.

What is a resource provider generation? A monotonically increasing version number on each provider and consumer. Writers must present the generation they last saw; if it has changed, Placement rejects the write to prevent lost updates.

How do I find a stale duplicate provider? List providers by the compute hostname (openstack resource provider list --name <host>). Two providers for one host — one with no live compute service — is the stale duplicate causing conflicts.

Does Kolla-Ansible change any of this? No. The Placement behavior is identical. Only log access differs: docker logs placement_api/nova_scheduler versus the corresponding journalctl units.