RabbitMQ Shovel & Federation Cross-Site Setup Prompt
Decide between shovel and federation and design a reliable cross-site or cross-cluster message-replication setup that survives WAN flakiness without duplicating or losing messages.
- Target user
- Platform and SRE engineers connecting RabbitMQ across sites or clusters
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior platform engineer who has connected RabbitMQ clusters across data centers using shovel and federation without creating message loops or losing messages over a flaky WAN. Help me design my cross-site setup. I will provide: - The topology: source and destination clusters/sites, direction of flow, and why (DR, geo-routing, migration) [DESCRIBE] - What needs to move: specific queues, or whole exchanges/topics [DESCRIBE] - Network reality: latency, reliability, TLS requirements between sites [DESCRIBE] - Current setup if any: `rabbitmqctl list_parameters` / `list_policies`, plugin status [PASTE OUTPUT] Your job: 1. **Choose shovel vs federation** — shovel for moving messages from a specific queue/source to a destination (point-to-point, simple, good for migrations and draining); federation for transparently linking exchanges or queues so consumers everywhere see messages. Justify the choice for my use case. 2. **Design for reliability** — use acknowledgements / `ack-mode = on-confirm` so a message is only removed from the source after the destination confirms it, set reconnect/`reconnect-delay`, and explain how the link behaves when the WAN drops and recovers. 3. **Prevent loops and duplication** — for bidirectional or multi-site federation, explain max-hops / loop detection and how to avoid a message ping-ponging between sites; flag where at-least-once delivery means consumers must be idempotent. 4. **Secure the link** — TLS between sites, restricted vhost/user for the shovel/federation connection, and least-privilege permissions. 5. **Operate it** — how to monitor shovel/link status (`list_shovels` / federation status), what a stuck or blocked link looks like, and how to drain and decommission a shovel cleanly after a migration. Output as: (a) shovel-vs-federation decision with reasoning, (b) the parameter/policy definitions, (c) reliability and loop-prevention settings, (d) monitoring and teardown steps. Stand the link up on a staging pair of brokers first. Use confirm-based ack-mode so the WAN dropping mid-transfer doesn't lose messages, and review carefully before pointing a shovel at a prod queue — a misconfigured shovel can drain or duplicate a live queue's messages.
Why this prompt works
Connecting RabbitMQ across sites is where the simple mental model of a single broker breaks down, and the first decision — shovel versus federation — sets the whole design. The prompt forces that choice with the right framing: shovel is point-to-point and ideal for migrations or draining a specific queue, while federation transparently links exchanges or queues so consumers everywhere see the traffic. Picking the wrong one leads to either fragile hand-rolled plumbing or unexpected fan-out.
Reliability over a WAN is the crux. The prompt insists on confirm-based ack-mode so a message leaves the source only after the destination confirms it, plus reconnect handling so the link self-heals when the network blips. That single setting is the difference between a DR link you can trust and one that silently drops messages every time the WAN hiccups. For multi-site federation it also raises loop prevention via max-hops, because a message looping endlessly between two federated sites is a classic and painful failure.
Finally it treats the link as something to operate, not just configure: TLS and a least-privilege user for the cross-site connection, monitoring for stuck links, and a clean teardown path after a migration. With staging validation and the explicit warning that a misconfigured shovel can drain or duplicate a live queue, the prompt keeps a cross-site setup from becoming a cross-site incident.