🚨 RabbitMQ Error Guide: 'handshake_timeout' — Fix Slow or Non-AMQP Connections

Overview

When a TCP connection reaches RabbitMQ’s AMQP listener, the client is expected to send the protocol header and complete the connection handshake (Connection.Start / Start-Ok / Tune / Open) within a bounded window — 10 seconds by default. If the client opens the socket but never finishes the handshake in time, the broker closes it with a handshake_timeout. This protects the broker from accumulating half-open connections that consume file descriptors and Erlang processes without ever becoming usable.

You will see it in the broker log:

=ERROR REPORT==== closing AMQP connection <0.2044.0> (10.0.6.12:0 -> 10.0.4.21:5672):
{handshake_timeout,handshake}

Note the client port 0 and the absence of any credentials — the connection never got far enough to authenticate. A variant appears during the TLS phase before AMQP even starts:

=ERROR REPORT==== closing AMQP connection (10.0.6.12:0 -> 10.0.4.21:5671):
{handshake_timeout,tls_handshake}

Critically, handshake_timeout usually is not your application clients — it’s most often something opening a TCP socket and not speaking AMQP: a load-balancer health check, a port scanner, a monitoring probe, or a client pointed at the wrong port or protocol.

Symptoms

• A steady trickle of handshake_timeout closures in the broker log, often from the same source IPs on a regular cadence.
• The offending log lines show peer port 0 and no user — the connection never authenticated.
• Real application clients connect fine; the timeouts come from infrastructure IPs (load balancers, monitoring hosts).
• After a TLS misconfiguration, a burst of {handshake_timeout,tls_handshake} on port 5671.

sudo grep -i 'handshake_timeout' /var/log/rabbitmq/rabbit@$(hostname -s).log | tail -5

=ERROR REPORT==== closing AMQP connection (10.0.6.12:0 -> 10.0.4.21:5672): {handshake_timeout,handshake}
=ERROR REPORT==== closing AMQP connection (10.0.6.13:0 -> 10.0.4.21:5672): {handshake_timeout,handshake}

Common Root Causes

1. A TCP-only health check on the AMQP port

The most common cause: a load balancer or orchestrator health check opens a TCP connection to 5672, confirms the port accepts, and closes — never sending the AMQP header. The broker waits the full handshake window, then logs a timeout.

sudo grep 'handshake_timeout' /var/log/rabbitmq/rabbit@$(hostname -s).log \
  | grep -oE '10\.[0-9.]+:0' | sort | uniq -c | sort -rn | head

    288 10.0.6.12:0
    287 10.0.6.13:0

Two source IPs producing near-identical counts on a fixed cadence is the signature of health-check probes.

2. A client speaking the wrong protocol or hitting the wrong port

An HTTP client, an MQTT/STOMP client, or a Prometheus scraper pointed at 5672 opens the socket but never sends a valid AMQP header, so the handshake never progresses.

rabbitmqctl list_connections name protocol state | grep -v running

Anything stuck in a pre-running state, or clients that appear and vanish without authenticating, points at a wrong-protocol connection.

3. Slow or misconfigured TLS negotiation

On the TLS port (5671), an expensive cipher, a large certificate chain, or a client stalling mid-negotiation can blow the TLS handshake window before AMQP starts.

sudo grep 'tls_handshake' /var/log/rabbitmq/rabbit@$(hostname -s).log | tail

=ERROR REPORT==== closing AMQP connection (10.0.6.20:0 -> 10.0.4.21:5671): {handshake_timeout,tls_handshake}

4. A network path so slow the handshake can’t complete

High latency, packet loss, or a saturated link between client and broker can stretch the handshake beyond the default window even for a legitimate AMQP client.

rabbitmqctl list_connections name peer_host recv_oct state | sort

A legitimate client with almost no bytes received before closure suggests the handshake stalled on the wire.

5. A handshake_timeout set too aggressively low

An operator lowered handshake_timeout in rabbitmq.conf and now trips slow-but-valid clients.

rabbitmqctl environment | grep -i handshake_timeout

{handshake_timeout,10000}

Diagnostic Workflow

Step 1: Confirm the closures and group by source

sudo grep 'handshake_timeout' \
  /var/log/rabbitmq/rabbit@$(hostname -s).log \
  | grep -oE '10\.[0-9.]+:0' | sort | uniq -c | sort -rn | head

A few IPs producing regular, high counts almost always means infrastructure probes, not application clients. Note which port (5672 vs 5671).

Step 2: Identify what those IPs are

Map the source IPs to your infrastructure — load balancer nodes, health-check subnets, monitoring hosts, or an unexpected scanner.

# reverse lookup / correlate against known infra
host 10.0.6.12

If they’re your load balancers, the fix is the probe configuration, not the broker.

Step 3: Separate benign probes from real client failures

rabbitmqctl list_connections name peer_host protocol state | grep -v ' running'

If real application IPs also show handshake failures, investigate network latency or TLS — those are genuine problems, not noise.

Step 4: Check the TLS phase if on port 5671

sudo grep 'tls_handshake' /var/log/rabbitmq/rabbit@$(hostname -s).log | tail
openssl s_client -connect <BROKER_HOST>:5671 -tls1_2 </dev/null

A slow or failing openssl s_client handshake reproduces the TLS-phase timeout outside your app.

Step 5: Fix the source, or tune the window as a last resort

Point health checks at a real probe, correct the client’s port/protocol, or fix the network/TLS path. Only if legitimate clients genuinely need longer should you raise handshake_timeout.

# verify the current setting before changing it
rabbitmqctl environment | grep -i handshake_timeout

Example Root Cause Analysis

An on-call engineer notices thousands of handshake_timeout lines per hour and worries the broker is under attack or failing. The application team insists their services connect fine — and they do. Grouping the log by source:

sudo grep 'handshake_timeout' /var/log/rabbitmq/rabbit@$(hostname -s).log \
  | grep -oE '10\.[0-9.]+:0' | sort | uniq -c | sort -rn | head

   1440 10.0.6.12:0
   1438 10.0.6.13:0

Exactly two IPs, near-identical counts, one closure roughly every five seconds each. Those IPs are the two nodes of the TCP load balancer fronting the cluster. The LB’s health check is a bare TCP connect to port 5672 — it opens a socket, sees the port is live, and closes without ever sending the AMQP protocol header. Each probe sits in the broker’s handshake queue until the 10-second window expires, then logs a timeout.

The fix is to stop probing AMQP with a bare TCP check. The team repoints the health check at the broker’s built-in health endpoint on the management port instead:

# load balancer health check
GET http://<node>:15672/api/health/checks/alarms   (expects 200)

After the change, the handshake_timeout flood stops, the log is quiet enough to see real errors, and the load balancer now detects an actually-unhealthy node (in alarm) rather than just an open port. The “attack” was the health check all along.

Prevention Best Practices

• Never health-check the AMQP port with a bare TCP connect — use the management HTTP health endpoints (/api/health/checks/...) so probes don’t pile up as handshake timeouts.
• Point clients at the correct port and protocol; keep AMQP (5672/5671), management (15672), and plugin ports (MQTT/STOMP) clearly separated and documented.
• Monitor TLS handshake cost on 5671 — prefer efficient ciphers and keep certificate chains lean so negotiation finishes comfortably within the window.
• Investigate latency/packet loss if legitimate application IPs (not infra) show handshake timeouts; the network path, not the broker, is usually at fault.
• Leave handshake_timeout at its default unless you can prove valid clients need longer — lowering it trips slow clients, and raising it masks probe noise.
• Alert on the rate and source of handshake_timeout so a real client-side regression stands out from steady health-check noise. The free incident assistant can group closures by source IP to separate probes from genuine failures. More in the RabbitMQ guides.

Quick Command Reference

# Confirm handshake_timeout closures
sudo grep -i 'handshake_timeout' /var/log/rabbitmq/rabbit@$(hostname -s).log | tail -10

# Group closures by source IP (peer port 0 = never authenticated)
sudo grep 'handshake_timeout' /var/log/rabbitmq/rabbit@$(hostname -s).log \
  | grep -oE '10\.[0-9.]+:0' | sort | uniq -c | sort -rn | head

# Connections not yet running (stuck mid-handshake)
rabbitmqctl list_connections name peer_host protocol state | grep -v ' running'

# TLS-phase timeouts on port 5671
sudo grep 'tls_handshake' /var/log/rabbitmq/rabbit@$(hostname -s).log | tail

# Current handshake_timeout setting
rabbitmqctl environment | grep -i handshake_timeout

Conclusion

A handshake_timeout closure means a TCP connection reached the broker but never completed the AMQP (or TLS) handshake within the allowed window, so RabbitMQ dropped the half-open socket. Peer port 0 and no username are the tell that it never authenticated. The usual root causes:

1. A bare TCP health check on the AMQP port that never speaks AMQP.
2. A client on the wrong port or using the wrong protocol.
3. Slow or misconfigured TLS negotiation on port 5671.
4. A network path too slow or lossy to finish the handshake in time.
5. A handshake_timeout tuned too aggressively low.

Group the closures by source IP first — a couple of infra IPs on a fixed cadence is almost always a health check, and the fix is the probe configuration, not the broker. Reserve tuning the timeout for genuinely slow legitimate clients.