Skip to content
DevOps AI ToolKit
Newsletter
RabbitMQ Troubleshooting Toolkit

RabbitMQ Troubleshooting Toolkit

Use this RabbitMQ troubleshooting toolkit to diagnose RPC timeouts, growing queue backlogs, missing consumers, memory and disk alarms, cluster partitions, and OpenStack oslo.messaging timeouts.

Paste your error and get a triage plan.

Paste a log line, CLI error, stack trace, service failure, or config snippet and get a structured troubleshooting plan. Your snippet is carried straight into the AI Incident Response Assistant with RabbitMQ context prefilled.

Open the RabbitMQ Incident Assistant → Browse RabbitMQ prompts

Do not paste secrets, tokens, private keys, passwords, or customer data. Your snippet stays in your browser until you open the assistant.

Top 25 RabbitMQ Errors and Failure Modes

The production failures engineers hit most — each links to a full cause → fix → prevention guide.

RabbitMQ RPC timeout

A remote procedure call over AMQP never receives its reply within the timeout window, usually because the reply queue, consumer, or broker is stalled.

Queue backlog growing

Messages accumulate faster than consumers can drain them, driving up queue depth, memory, and end-to-end latency.

Messages unacknowledged

Delivered messages sit in the unacked state because consumers never ack them, often due to slow processing or an oversized prefetch count.

Consumers missing

A queue has no active consumers after a cancel notification or crash, so messages pile up with nothing to process them.

Memory alarm

RabbitMQ raises a memory resource alarm and blocks publishers once broker memory use crosses the configured high watermark.

Disk free alarm

A free-disk-space resource alarm triggers when available disk drops below the threshold, pausing publishers to protect the node.

Node down

A cluster node is unreachable or stopped, leaving its home queues unavailable until the node recovers or queues fail over.

Cluster partition

A network partition splits the Mnesia cluster into isolated islands that disagree on state and must be healed carefully.

Connection churn

Clients repeatedly open and drop connections, frequently from missed heartbeats or short-lived connections, overloading the broker.

Channel closed

The broker closes an AMQP channel after a protocol or precondition violation, breaking any publishes or consumes on it.

Authentication failed

A login is refused because the username, password, or credential mechanism does not match the broker's configured users.

Vhost permission denied

A user is not authorized to access a virtual host or resource because configure, write, or read permissions are missing.

Queue not found

An operation references a queue that does not exist on the vhost, returning a not-found error to the client.

Exchange not found

A publish or bind targets an exchange that has not been declared, causing a not-found channel exception.

Publisher confirms timeout

The broker does not return an ack for published messages in time, leaving the publisher unsure whether messages were persisted.

Consumer timeout

A consumer exceeds the delivery acknowledgement timeout, so RabbitMQ closes the channel and requeues the in-flight message.

Mirrored queue sync issue

A classic mirrored queue mirror fails to synchronize with its master, risking message loss on failover.

Quorum queue issue

A quorum queue cannot elect or reach a Raft leader, so it stops accepting or delivering messages until majority is restored.

Stream queue growth

A stream's on-disk log grows unbounded when retention limits are unset, consuming disk faster than expected.

RabbitMQ management UI unavailable

The management plugin listener fails to start or bind, so the web UI and HTTP API become unreachable.

Erlang cookie mismatch

Nodes fail to cluster because their Erlang cookies differ, blocking the distributed authentication handshake.

TLS handshake failure

A client or peer connection fails during the TLS handshake because of certificate, cipher, or protocol version mismatches.

OpenStack oslo.messaging timeout

An OpenStack service times out waiting on an oslo.messaging RPC reply carried over RabbitMQ, surfacing as a MessagingTimeout.

Cinder scheduler RPC timeout

Cinder volume operations fail when the scheduler's RPC call over RabbitMQ times out, usually from broker backlog or a stuck reply queue.

Neutron RPC timeout

Neutron agents time out on RPC calls to the server over RabbitMQ, breaking port, network, or L3 provisioning.

Validate your config before you redeploy

Catch the structural mistakes that cause outages — 100% in your browser, nothing uploaded.

YAML Validator (config) Not sure? Start with the Incident Assistant →

RabbitMQ Troubleshooting Hubs

Fix a whole class of RabbitMQ errors at once — each hub gathers the related guides for one failure class.

RabbitMQ Connection, Channel & Auth Errors

Fix RabbitMQ connection and channel failures — ECONNREFUSED on 5672, ACCESS_REFUSED logins, TLS handshake failures, missed heartbeats, frame and protocol errors, blocked connections, and too-many-channels — with the exact rabbitmqctl and openssl commands.

Open hub

RabbitMQ Cluster, Queue & Resource Errors

Fix RabbitMQ cluster and queue failures — Mnesia network partitions, quorum queues that lose majority, nodes that will not rejoin, memory/disk high-watermark alarms, flow control, unroutable messages, and stuck consumers — with exact rabbitmqctl and rabbitmq-diagnostics commands.

Open hub

Best RabbitMQ Prompts

Turn symptoms, logs, and config into a structured plan with prompts tuned for RabbitMQ.

RabbitMQ oslo.messaging RPC timeout debug

Triage OpenStack RPC timeouts over RabbitMQ

Use this prompt

oslo.messaging RabbitMQ backlog triage

Clear OpenStack queue backlogs on the broker

Use this prompt

RabbitMQ queue backpressure & flow control triage

Diagnose growing queues and flow control

Use this prompt

RabbitMQ memory & disk alarm resource-limit triage

Resolve memory and disk resource alarms

Use this prompt

RabbitMQ cluster partition recovery

Recover safely from a network partition

Use this prompt

RabbitMQ heartbeat & connection churn triage

Debug missed heartbeats and reconnect storms

Use this prompt

RabbitMQ queue investigation

Root-cause an unhealthy or backed-up queue

Use this prompt

RabbitMQ consumer prefetch & QoS tuning

Tune prefetch to fix unacked message buildup

Use this prompt

Download the RabbitMQ Troubleshooting Runbook Pack

Queue-level runbooks for a broker that is backing up, partitioned, or alarmed — plus OpenStack oslo.messaging RPC timeouts.

  • Cluster status & partition checks
  • Queue depth & consumer diagnosis
  • Memory / disk alarm response
  • Missed-heartbeat troubleshooting
  • oslo.messaging (OpenStack) RPC timeout
Download the RabbitMQ Runbook Pack

All RabbitMQ Troubleshooting Guides

Grouped by failure-mode type — each guide covers cause, fix, validation, and prevention.

Authentication & TLS (10)

RabbitMQ Error: STOMP plugin CONNECT error / authentication failed access to vhost refused for user No compatible authentication mechanism Authentication failed (rejected by the remote node), please check the Erlang cookie HTTP access denied certificate verify failed TLS handshake failed ACCESS_REFUSED - access to queue refused for user PLAIN login refused: user does not exist ACCESS_REFUSED

Networking (23)

RabbitMQ Error: AMQP 1.0 connection failed / plugin not enabled RabbitMQ Error: 'connection.blocked' — Publishers Blocked by a Resource Alarm RabbitMQ Error: Federation Link Down / Upstream Not Running RabbitMQ Error: MQTT plugin connection refused / not enabled RabbitMQ Error: Shovel 'connection_refused' Cannot Reach Source or Destination RabbitMQ Error: vm_memory_high_watermark alarm set, connections blocked RabbitMQ Error: 'WebSocket connection failed' with the Web-STOMP Plugin CONNECTION_FORCED handshake_timeout closing AMQP connection ... {bad_header,...} Failed to start Ranch listener ... eaddrinuse {socket_error, epipe} Node rabbit@host is down {socket_error, econnreset} federation link ... unavailable flow Error on AMQP connection Shovel ... terminated ACCESS_REFUSED - vhost not found connection_closed_abruptly {inet_error,etimedout} ECONNREFUSED Mnesia network partition

Storage (1)

RabbitMQ Error: Lazy Queue Mode Deprecated, High Disk I/O After Conversion

Deployment & Builds (2)

feature flag is disabled Blue-Green RabbitMQ Upgrades With AI

Runtime (3)

RabbitMQ Error: Dead-Letter Exchange Not Found, Messages Not Dead-Lettering RabbitMQ Error: x-max-length Reached, Messages Dropped or Dead-Lettered {error, queue_process_is_stopped}

Performance (8)

RabbitMQ Error: rabbitmq_prometheus /metrics Scrape Timeout Under Load RabbitMQ Error: Stream Publisher Confirms Timing Out consumer_timeout Error: operation ... timed out ra: command timeout timeout_waiting_for_tables CHANNEL_ERROR missed heartbeats

API errors (3)

RabbitMQ Error: 'khepri_db' Metadata Store Enable and Replication Failure Mnesia is overloaded Automating RabbitMQ With the Management API and AI

Configuration (4)

RabbitMQ Error: Stream Consumer 'offset out of range' / Invalid Offset Spec Error preparing configuration PRECONDITION_FAILED - unknown delivery tag unknown exchange type x-delayed-message

Other (36)

RabbitMQ Error: Classic Mirrored Queue Synchronisation Stuck / Unsynchronised Mirrors RabbitMQ Error: 'classic_queue_mirroring is deprecated' HA Policy Rejected RabbitMQ Error: 'failed to import definitions' From definitions.json on Boot RabbitMQ Error: Delayed Message Exchange Plugin Not Delaying Messages RabbitMQ Error: Messages Disappearing Due to Message TTL Expiry RabbitMQ Error: Policy Not Applied to a Queue Because a Higher-Priority Policy Matches RabbitMQ Error: Quorum/Stream Queue Leaders Not Balancing Across Nodes RabbitMQ Error: 'reject-publish' queue overflow returning basic.nack to publishers RabbitMQ Error: Single Active Consumer Not Failing Over to the Next Consumer RabbitMQ Error: Unacked Messages Stuck and Not Redelivered BOOT FAILED message size exceeds max_message_size plugins_not_found STREAM_NOT_AVAILABLE UNEXPECTED_FRAME frame_too_large NOT_FOUND - no exchange CHANNEL_ERROR - expected channel.open consumer cancelled CRASH REPORT ... gen_server terminated PRECONDITION_FAILED - inequivalent arg for exchange unable to join cluster inconsistent_cluster Failed to start Ranch listener statistics database could not be contacted basic.nack operation queue.declare caused a channel exception home node ... is down cannot reach majority RESOURCE_LOCKED - cannot obtain exclusive access Discarding message in an old incarnation epmd error for host ... nxdomain file descriptor limit alarm set quorum queue ... no leader elected Sharding RabbitMQ With the Consistent Hash Exchange and AI NO_ROUTE
Browse the full RabbitMQ category

RabbitMQ troubleshooting FAQ

How do I fix a RabbitMQ RPC timeout?
Confirm the broker is healthy and not under a resource alarm, check that the reply queue has an active consumer, and measure queue depth and consumer ack latency. RPC timeouts almost always trace back to a stalled consumer, a broker backlog, or a network hiccup rather than the timeout value itself, so fix the slow or missing consumer before raising the timeout.
Why is my RabbitMQ queue backing up?
A queue grows when the publish rate exceeds the effective consume rate. Common causes are too few or crashed consumers, an oversized prefetch that leaves many messages unacknowledged, slow downstream processing, or flow control kicking in from a resource alarm. Compare incoming vs deliver/ack rates and the unacked count in the management UI to find the bottleneck.
What triggers a RabbitMQ memory alarm?
RabbitMQ raises a memory alarm when total broker memory use crosses the high watermark (40% of system RAM by default), at which point it blocks publishers to protect the node. It is usually driven by deep queues holding messages in memory, many connections and channels, or the stats database. Drain the backlog, switch large queues to lazy or quorum/stream types, and add memory or consumers to clear it.
How do I recover from a RabbitMQ cluster partition?
First identify which side holds the authoritative state, then pick a partition-handling strategy (pause_minority, autoheal, or manual). For a manual recovery, stop the losing nodes, restart them so they rejoin and re-sync from the winning side, and verify queues and bindings converged. Prevent recurrence with a stable network, quorum queues, and an explicit cluster_partition_handling policy.
How does RabbitMQ relate to OpenStack oslo.messaging timeouts?
OpenStack services use oslo.messaging with RabbitMQ as the default transport for RPC and notifications, so a MessagingTimeout in Nova, Cinder, or Neutron usually means the underlying RabbitMQ call did not get a reply in time. Investigate the broker for backlog, resource alarms, missed heartbeats, or partitions, and confirm the services' reply queues have live consumers before tuning oslo.messaging timeout values.
What causes RabbitMQ authentication and permission failures?
Login refusals come from a wrong username or password, a disabled or deleted user, or an auth mechanism the client and broker do not share. Once authenticated, a user can still be blocked by missing configure/write/read permissions on the target vhost. Check the user exists, its permissions on the correct vhost, and that the connecting vhost matches what the application expects.