🌐 RabbitMQ Cross-Site Federation and Shovel With AI

The first time I connected two RabbitMQ clusters across regions, I reached for federation because it was the first thing I read about, wired up a federated exchange, and then spent a day confused about why messages were looping and duplicating between sites. The tool was fine. I’d just picked the wrong one for what I actually wanted, which was a simple one-directional message move from a source queue in one region to a sink in another — a textbook shovel job. Federation and shovel both move messages between brokers, and choosing the wrong one creates problems that look like bugs but are really design mismatches.

This is solid AI territory because the choice between federation and shovel is a decision tree with clear rules, and the configuration for both is verbose and easy to typo. The model is good at asking “what are you actually trying to do?” implicitly and routing you to the right tool, and good at generating the policy and parameter blocks correctly. What it can’t do is reach across your network and confirm the link actually establishes through your firewalls and TLS — and cross-site links fail at exactly that layer more often than anywhere else.

Describe the topology and direction, let the AI pick the tool

Don’t ask “should I use federation or shovel?” in the abstract. Describe the sites, the direction, and whether you’re linking exchanges or moving specific queues.

I have two RabbitMQ clusters: us-east (primary) and eu-west (secondary). I want every message published to the events exchange in us-east to also be available to consumers in eu-west, continuously, surviving broker restarts. Separately, I have a one-off need to drain a stuck migration queue from an old broker into a new one, once. For each case, tell me whether to use federation or shovel, and why. Then give me the configuration.

The clean answer: federation for the continuous exchange replication (it’s built to link exchanges/queues with loop prevention across many nodes), shovel for the point-to-point drain (it’s a simple, reliable “take from here, put there” worker, ideal for one-shot or queue-to-queue moves). If the AI reaches for federation on the one-off drain, that’s the mistake I made — push back and ask why a shovel isn’t simpler.

Federation: link the exchanges, mind the loops

Federation is policy-driven. You define an upstream and apply a policy that says “federate exchanges matching this pattern from that upstream.”

# On eu-west: point at us-east as an upstream
rabbitmqctl set_parameter federation-upstream us-east \
  '{"uri":"amqps://federation_user:secret@us-east.example.com:5671","expires":3600000}'

# Federate the events exchange from that upstream
rabbitmqctl set_policy federate-events "^events$" \
  '{"federation-upstream":"us-east"}' --apply-to exchanges

# Confirm the link came up
rabbitmqctl eval 'rabbit_federation_status:status().'

That eval status command is what tells you the link is actually running and not stuck starting because the URI or credentials are wrong. Federation has built-in loop prevention via message headers, but only within the federation mechanism — if you also set up a reverse link by hand, you can still create duplicates. Ask the AI: “Could this configuration create a message loop between the two sites?” and have it trace the direction.

Shovel: move a specific queue, point to point

Shovel is even simpler — source, destination, done. For the one-off drain, a dynamic shovel that deletes itself when the source is empty is exactly right.

rabbitmqctl set_parameter shovel migrate-old \
  '{"src-uri":"amqp://user:secret@old-broker.example.com",
    "src-queue":"migration",
    "dest-uri":"amqp://user:secret@new-broker.example.com",
    "dest-queue":"migration",
    "src-delete-after":"queue-length"}'

# Watch it run and confirm it drains
rabbitmqctl shovel_status

src-delete-after: queue-length makes the shovel stop and remove itself once it has moved the messages that were present when it started — perfect for a one-shot migration. The AI should suggest this for a drain rather than a permanent shovel that lingers forever.

The verification AI cannot do for you

Cross-site links fail at the network and TLS boundary, and no amount of correct config helps if port 5671 is firewalled or the cert chain doesn’t validate. Prove the link end to end on staging brokers before you trust it.

# From the downstream node, can you even reach the upstream's AMQP port?
nc -zv us-east.example.com 5671

# Establish the link, then publish at the source and confirm arrival
rabbitmqadmin -H us-east.example.com publish exchange=events \
  routing_key=test payload='{"probe":1}'
rabbitmqadmin -H eu-west.example.com get queue=events.local count=1

The end-to-end probe — publish at the source, consume at the destination — is the only thing that proves the federation or shovel link genuinely carries messages across the boundary. I’ve had links report running while messages didn’t flow because a downstream binding was missing. Trust the probe, not the status line alone.

Where AI overreaches

It sometimes proposes federation for a simple queue drain because federation is the “fancier” feature. Resist that — shovel is more reliable for point-to-point and far easier to reason about. It also tends to put credentials inline in the URI without mentioning TLS; for cross-site links over the public internet, push it: “Use amqps with TLS and tell me how to verify the cert chain, not plaintext amqp.” A link carrying your messages across regions in cleartext is a problem the model won’t flag unless you ask.

It can also under-specify the upstream expires and max-hops settings, which matter for cleanup and loop bounding in multi-cluster meshes. Ask it to set max-hops explicitly when more than two sites are involved.

My loop

Describe the sites and direction, let the AI pick federation versus shovel and justify it, take the generated parameter and policy blocks, and apply them to staging brokers. Then I run the network reachability check and an end-to-end publish/consume probe across the boundary before anything touches production. The AI routes the decision and writes the verbose config correctly; the cross-site probe through real firewalls and TLS is mine to run.

These federation and shovel prompts live with my other prompts, and the RabbitMQ category covers the quorum-queue and confirm patterns that keep cross-site message delivery durable.