RabbitMQ Cluster, Queue & Resource Errors
The failures that degrade the broker itself rather than a single connection: a Mnesia network partition (split brain), a quorum queue that cannot elect a leader, a node that will not rejoin the cluster, memory or disk high-watermark alarms that block publishers, flow control kicking in, messages that are unroutable or dead-lettered, and streams or consumers that stall. Each guide opens with the direct cause and the `rabbitmqctl`, `rabbitmq-diagnostics`, and `rabbitmq-queues` commands to confirm it. Start with your symptom below, or paste the broker log into the assistant.
Fix your error now
Paste the error or logs and let the Incident Assistant diagnose it, or run a guided triage prompt with AI — no copy-paste.
Every guide in this cluster
64 guides-
'Mnesia network partition' Cluster Split Brain
Fix RabbitMQ Mnesia network partitions (split brain): detect partitioned nodes, choose a partition handling strategy, and safely recover a clustered broker.
-
'cannot reach majority' Quorum Queue Lost Quorum
Fix RabbitMQ quorum queues that lost majority: diagnose 'cannot reach majority', under-replicated members, and recover queues after losing over half the nodes.
-
vm_memory_high_watermark alarm set, connections blocked
Fix RabbitMQ memory alarm blocking publishers: diagnose vm_memory_high_watermark, connection.blocked, memory_high_watermark_set, tune the watermark and drain queues.
-
'Node rabbit@host is down' Cluster Member Unreachable
Fix RabbitMQ 'Node rabbit@host is down' and 'not responding' errors: a crashed beam, stopped service, or blocked distribution ports. Diagnose and recover safely.
-
'{error, queue_process_is_stopped}' Queue Process Down
Fix RabbitMQ queue_process_is_stopped errors: classic, mirrored, or quorum queue process down on a failed node, failed delete_queue, and stuck operations.
-
'ACCESS_REFUSED - vhost not found' No Access to Virtual Host
Fix RabbitMQ vhost not found / no access to vhost errors: create the missing virtual host, grant per-vhost permissions, and correct URL-encoded vhost paths in clients.
-
'Authentication failed (rejected by the remote node), please check the Erlang cookie' — Fix Cookie Mismatch
Fix the RabbitMQ Erlang cookie mismatch that blocks rabbitmqctl and cluster joins: locate the .erlang.cookie file, align its value and permissions across nodes, and restart cleanly.
-
'basic.nack' Publisher Confirm Negative Acknowledgement
Fix RabbitMQ publisher nacks: diagnose basic.nack received under publisher confirms from failed persistence, leader failover, or resource alarms.
-
'BOOT FAILED' — Fix a Broker That Won't Start
Fix 'BOOT FAILED' in RabbitMQ: read the failed boot step, recover a corrupt schema or config, fix permissions, and get the broker to start cleanly.
-
'consumer cancelled' basic.cancel Notification
Fix RabbitMQ consumer cancel notifications: diagnose why basic.cancel is pushed to clients when a queue is deleted, its node fails, or it is recovered.
-
'consumer_timeout' Delivery Acknowledgement Timed Out
Fix RabbitMQ consumer_timeout: diagnose 'delivery acknowledgement timed out' channel closures from long processing or unacked messages and tune the limit.
-
'CRASH REPORT ... gen_server terminated' Reading Erlang Crashes
Read RabbitMQ Erlang crash reports: decode gen_server terminated and supervisor reports with noproc, badmatch, function_clause, case_clause, and badarg reasons.
-
'Discarding message in an old incarnation' Stale Node Reference
Fix 'Discarding message in an old incarnation' in RabbitMQ: node restarts, partition recovery, stale process references, and mirrored queue leftovers.
-
'Error preparing configuration' — Fix rabbitmq.conf Syntax
Fix 'error preparing configuration' in RabbitMQ: correct rabbitmq.conf syntax, resolve Cuttlefish schema errors, and fix advanced.config Erlang terms.
-
operation ... timed out' rabbitmqctl RPC Timeout
Fix RabbitMQ operation timed out errors from rabbitmqctl and cluster ops: overloaded nodes, slow internal RPC, long-running queries, and the --timeout flag.
-
'Failed to start Ranch listener' Management Plugin on 15672
Fix RabbitMQ management listener startup failures on port 15672: eaddrinuse port conflicts, the plugin not enabled, and bad listener config blocking the UI.
-
'feature flag is disabled' — Fix Blocked Declarations & Upgrades
Fix 'feature flag is disabled' in RabbitMQ: enable required feature flags, finish a cluster-wide upgrade first, and unblock quorum and stream declarations.
-
'federation link ... unavailable' Upstream Connection Failure
Fix RabbitMQ federation upstream unavailable errors: broken upstream connections, bad credentials or URIs, network blocks, and misconfigured upstream-sets and policies.
-
'file descriptor limit alarm set' FD Exhaustion
Fix the file descriptor limit alarm: raise ulimit and systemd LimitNOFILE, find connection and socket leaks, and clear the high watermark block.
-
'flow' Connection State Internal Flow Control
Fix RabbitMQ publishers stuck in 'flow' state: diagnose internal credit-based flow control from slow queues, disk, or CPU, distinct from resource alarms.
-
'home node ... is down' Classic Queue Unavailable
Fix RabbitMQ 'home node is down' errors: classic queue unavailable because its home/leader node is offline, queue shown as down in management, and HA fixes.
-
'inconsistent_cluster' Node Disagrees on Membership
Fix RabbitMQ inconsistent_cluster errors where a node thinks it's clustered with a peer that disagrees. Caused by stale state after a reset or forget. Recover safely.
-
'message size exceeds max_message_size' — Fix Oversized Messages
Fix 'message size larger than configured max' in RabbitMQ: raise max_message_size safely, shrink payloads, and use a claim-check for large blobs.
-
'Mnesia is overloaded' Metadata Churn Warning
Fix RabbitMQ 'Mnesia is overloaded' dump_log write_threshold warnings caused by queue churn, exclusive/auto-delete storms, binding churn and slow disk.
-
'NO_ROUTE' Mandatory Message Returned Unroutable
Fix RabbitMQ NO_ROUTE returned messages: diagnose missing bindings, wrong routing keys, default vs named exchange confusion, and exchange type mismatches.
-
'NOT_FOUND - no exchange' basic.publish Failures
Fix RabbitMQ publish failures: 404 NOT_FOUND on basic.publish, returned messages, channel closed on publish, and missing publisher confirms explained.
-
'NOT_FOUND' No Queue or Exchange (404)
Fix RabbitMQ NOT_FOUND channel errors: diagnose missing queues and exchanges, wrong vhost, auto-deleted queues, typos, and declaration order race conditions.
-
'operation queue.declare caused a channel exception' Declare Failures
Fix RabbitMQ queue.declare and queue.delete channel exceptions: x-queue-type mismatch, invalid arguments, passive declare on a missing queue, and delete-if-unused.
-
'plugins_not_found' — Fix Plugin Enable Failures
Fix 'plugins_not_found' in RabbitMQ: correct the plugin name, match its version to the broker, fix the plugins directory, and re-enable cleanly.
-
'PRECONDITION_FAILED - inequivalent arg for exchange' Declaration Conflict
Fix RabbitMQ PRECONDITION_FAILED inequivalent arg errors on exchange declare or delete: mismatched type, durable, auto-delete, and internal flags.
-
'PRECONDITION_FAILED - unknown delivery tag' Ack Failure
Fix RabbitMQ 'unknown delivery tag' errors: acking on the wrong channel, double-ack, acking after auto-ack, and stale delivery tags after reconnect.
-
'PRECONDITION_FAILED' Inequivalent Arg for Queue
Fix RabbitMQ PRECONDITION_FAILED inequivalent arg errors: reconcile durable, auto-delete, x-message-ttl, x-queue-type, and DLX mismatches on redeclared queues.
-
'quorum queue ... no leader elected' Raft Quorum Loss
Fix RabbitMQ quorum queues with no leader: down replicas, lost majority, partitioned Ra members, and stalled Raft leader elections.
-
'ra: command timeout' Quorum Queue Raft Timeout
Fix RabbitMQ quorum queue Raft timeouts: diagnose 'ra command timeout' and 'failed to start Raft' from slow disk, overloaded nodes, and network latency.
-
'resource alarm' Memory/Disk Free Limit Reached
Fix RabbitMQ resource alarms blocking publishers: diagnose memory high-watermark and disk free limit, find the queue consuming RAM, and clear the alarm safely.
-
'RESOURCE_LOCKED - cannot obtain exclusive access' Locked Queue
Fix RabbitMQ RESOURCE_LOCKED errors: exclusive queue owned by another connection, reconnect races, and 'cannot obtain exclusive access to locked queue' fixes.
-
'Shovel ... terminated' Worker Failure and Lost Upstream
Fix RabbitMQ shovel worker terminated and failed-to-start errors: bad source/destination URIs, wrong credentials, missing queues, and dynamic vs static shovel config.
-
'statistics database could not be contacted' Metrics Failure
Fix RabbitMQ statistics database unavailable and metrics timeout errors: overloaded stats collector, rates mode, and large topologies stalling the management UI.
-
'STREAM_NOT_AVAILABLE' — Fix Stream Leader Unavailability
Fix 'STREAM_NOT_AVAILABLE' in RabbitMQ Streams: restore a stream leader, recover replica quorum, enable the stream plugin and port, and reconnect clients.
-
'timeout_waiting_for_tables' Cluster Startup Failure
Fix RabbitMQ timeout_waiting_for_tables on startup: node boot order, last-disc-node down, Mnesia table sync, and forget_cluster_node recovery.
-
'unable to join cluster' Cookie and Version Mismatch
Fix RabbitMQ 'unable to join cluster' errors: Erlang cookie mismatch, cluster_name mismatch, RabbitMQ/Erlang version skew, and 'already a member' on join_cluster.
-
'UNEXPECTED_FRAME' — Fix AMQP Protocol Violations
Fix 'UNEXPECTED_FRAME' in RabbitMQ: stop sharing a channel across threads, keep frames in order, and upgrade buggy clients that violate AMQP framing.
-
'unknown exchange type x-delayed-message' Missing Plugin
Fix RabbitMQ unknown exchange type errors for x-delayed-message and x-consistent-hash: the exchange-type plugin is missing or disabled. Enable and verify it.
-
'classic_queue_mirroring is deprecated' HA Policy Rejected
Fix RabbitMQ's deprecated classic mirrored queue (ha-mode) policy warning and removal in 4.x: diagnose classic_queue_mirroring usage and migrate to quorum queues.
-
'failed to import definitions' From definitions.json on Boot
Fix RabbitMQ 'failed to import definitions' at startup: bad definitions.json path, invalid JSON, schema/validation errors, and missing vhosts or users blocking a clean boot.
-
'khepri_db' Metadata Store Enable and Replication Failure
Fix RabbitMQ Khepri metadata store errors: timed-out leader election, feature flag stuck state_changing, and Ra replication failures when enabling khepri_db across a cluster.
-
'reject-publish' queue overflow returning basic.nack to publishers
Fix RabbitMQ x-overflow reject-publish nacks: diagnose max-length/max-length-bytes limits, confirm publisher nacks, tune queue overflow, and stop dropped messages.
-
Classic Mirrored Queue Synchronisation Stuck / Unsynchronised Mirrors
Fix a RabbitMQ classic mirrored queue stuck with unsynchronised mirrors: trigger and monitor sync, understand ha-sync-mode, and migrate to quorum queues to end the synchronisation problem for good.
-
Dead-Letter Exchange Not Found, Messages Not Dead-Lettering
Fix RabbitMQ messages not routing to a dead-letter exchange: diagnose a missing x-dead-letter-exchange, unrouted DLX messages, and no-route drops with rabbitmqctl.
-
Delayed Message Exchange Plugin Not Delaying Messages
Fix rabbitmq_delayed_message_exchange not delaying: enable the plugin, declare x-delayed-message with x-delayed-type, set the x-delay header, and avoid the disk-persistence gotchas.
-
Federation Link Down / Upstream Not Running
Fix a RabbitMQ federation link stuck 'down': diagnose missing upstreams, unmatched policies, bad upstream URIs and blocked ports so exchange and queue federation links reach the running state again.
-
Lazy Queue Mode Deprecated, High Disk I/O After Conversion
Fix RabbitMQ lazy queue mode issues: diagnose deprecated x-queue-mode lazy, classic queue v2 behaviour, and high disk I/O after conversion with rabbitmqctl and diagnostics.
-
Messages Disappearing Due to Message TTL Expiry
Fix RabbitMQ messages vanishing from a queue: diagnose per-message and per-queue x-message-ttl expiry, expired messages dropped or dead-lettered, with rabbitmqctl.
-
MQTT plugin connection refused / not enabled
Fix RabbitMQ MQTT connection refused: enable rabbitmq_mqtt, open port 1883/8883, fix CONNACK auth (bad username or password), vhost mapping and anonymous logins.
-
Policy Not Applied to a Queue Because a Higher-Priority Policy Matches
Fix RabbitMQ policies that never apply to a queue: diagnose priority precedence, apply-to scope, and pattern overlap so the right effective_policy_definition lands via rabbitmqctl.
-
Quorum/Stream Queue Leaders Not Balancing Across Nodes
Fix RabbitMQ leaders piling on one node: set queue-leader-locator to balanced, rebalance quorum/stream leaders with rabbitmq-queues, and stop client-local leader placement.
-
rabbitmq_prometheus /metrics Scrape Timeout Under Load
Fix RabbitMQ Prometheus scrape timeouts: the /metrics endpoint stalls under load with many queues. Use per-object vs aggregated metrics, raise scrape_timeout, and shard collection.
-
Shovel 'connection_refused' Cannot Reach Source or Destination
Fix a RabbitMQ shovel stuck with 'connection_refused': diagnose bad URIs, blocked ports, wrong credentials and TLS so a dynamic or static shovel reconnects to its source or destination broker.
-
Single Active Consumer Not Failing Over to the Next Consumer
Fix RabbitMQ single active consumer not switching: enable x-single-active-consumer at declare time, cancel/ack correctly, and confirm failover with rabbitmqctl list_consumers.
-
STOMP plugin CONNECT error / authentication failed
Fix RabbitMQ STOMP CONNECT errors: enable rabbitmq_stomp, open port 61613, resolve ERROR frames for bad login, vhost access, heartbeats and destination prefixes.
-
Stream Consumer 'offset out of range' / Invalid Offset Spec
Fix a RabbitMQ stream consumer failing with 'offset out of range' or an invalid offset spec: understand retention-trimmed offsets, first/last/next/timestamp specs and how to attach safely.
-
Stream Publisher Confirms Timing Out
Fix RabbitMQ stream publisher confirms that never arrive: diagnose resource alarms, slow disk on the osiris log, under-replicated stream members and client timeouts so confirms flow again.
-
Unacked Messages Stuck and Not Redelivered
Fix RabbitMQ messages stuck in unacked state and never redelivered: a consumer holding them without acking. Diagnose prefetch, missing acks, and force redelivery safely.
-
x-max-length Reached, Messages Dropped or Dead-Lettered
Fix RabbitMQ dropping messages when x-max-length is reached: diagnose max-length overflow, drop-head vs reject-publish, and reason 'maxlen' dead-lettering with rabbitmqctl.
RabbitMQ AI prompts
Copy-paste, production-safe prompts for this stack.
RabbitMQ command center
Top errors, validators, and runbooks for the whole stack.
Frequently asked questions
How do I recover from a RabbitMQ Mnesia network partition (split brain)?
rabbitmq-diagnostics cluster_status and review your cluster_partition_handling mode. See Mnesia network partition.