⏳ RabbitMQ Message TTL and Expiration Strategy With AI

A teammate once set a 60-second message TTL on a queue to “keep it from filling up,” and for months it worked beautifully — until a consumer deploy took 90 seconds and the queue silently expired every message in flight during the rollout. No error, no dead-letter, just gone. That’s the thing about TTL in RabbitMQ: it’s a deletion mechanism dressed up as a convenience, and the failure mode is silent data loss precisely when you’re already having a bad day.

Designing an expiration strategy is a good fit for AI because the rules are fiddly and counterintuitive — per-message versus per-queue TTL behave differently, head-of-line ordering matters, and expired messages can either vanish or dead-letter depending on configuration. The model knows these rules and is good at turning “I want messages to expire after X, but I don’t want to lose anything important” into a concrete policy. What it can’t do is decide for you which messages are safe to drop — that’s a business call — and it can’t watch your staging queue to confirm nothing important expired during a slow deploy.

Tell the AI what’s safe to lose, and what isn’t

The first question isn’t “how do I set a TTL” — it’s “which messages can I afford to expire?” Frame it that way and the design follows.

I have a RabbitMQ queue of real-time price-update events. They’re only useful for about 30 seconds — after that a newer update has superseded them, so expiring stale ones is fine. But I also have a separate queue of order events that must NEVER silently expire. Design a TTL strategy for each. For the price queue, should I use per-queue or per-message TTL? For the order queue, how do I make sure nothing expires, and if I do want an expiration safety net there, how do I avoid silent loss?

A good answer separates them cleanly: per-queue x-message-ttl for the price queue since all messages share the same staleness window, and for the order queue, no TTL — or, if you insist on a safety net, a TTL that dead-letters to a parked queue rather than dropping. The principle the AI should state plainly: expiration without dead-lettering is silent deletion.

The per-queue vs per-message distinction the AI nails

This is the rule people get wrong, and the model explains it well:

Per-queue TTL (x-message-ttl as a queue argument) applies uniformly and expires messages strictly from the head of the queue — a message only expires when it reaches the front, so a slow consumer can hold expired messages in place behind it. Per-message TTL (set in message properties) lets each message have its own lifetime, but because RabbitMQ only checks the message at the head, a message with a short TTL stuck behind one with a long TTL won’t be discarded until it reaches the front. Either way, expiration is a head-of-line operation, not a background sweep.

That head-of-line detail is the whole game. People assume there’s a background reaper deleting expired messages everywhere in the queue. There isn’t — for classic queues, only the head gets checked. A long-TTL message at the front pins short-TTL messages behind it.

Configure it so expiry dead-letters, not vanishes

For anything you can’t afford to lose silently, route expired messages to a dead-letter queue. Now “expired” becomes “parked for review” instead of “deleted.”

# Price queue: short TTL, fine to drop (fanout of fresh data behind it)
rabbitmqadmin declare queue name=prices durable=true \
  arguments='{"x-message-ttl":30000}'

# Orders queue: TTL safety net that DEAD-LETTERS instead of dropping
rabbitmqadmin declare exchange name=orders.expired type=fanout durable=true
rabbitmqadmin declare queue name=orders.expired.parked durable=true
rabbitmqadmin declare binding source=orders.expired \
  destination=orders.expired.parked

rabbitmqadmin declare queue name=orders durable=true \
  arguments='{"x-message-ttl":86400000,"x-dead-letter-exchange":"orders.expired"}'

Now an order that somehow ages past 24 hours lands in orders.expired.parked where an alert can fire, instead of disappearing. Ask the AI to default to this pattern for important queues: “Make expiration observable — route expired messages somewhere I can see them.”

Validate against the slow-consumer scenario

This is the test that catches the deploy-day disaster. On staging, fill the queue, stop the consumer for longer than the TTL, then restart it and check what survived.

# Seed messages, then simulate a slow/stopped consumer
rabbitmqadmin publish exchange="" routing_key=prices payload='{"p":100}'
# ... stop consumer longer than the 30s TTL ...

# How many expired? Watch the ready count drop on its own
rabbitmqadmin list queues name messages_ready messages

# For the orders queue, confirm nothing landed in parked
# unless it genuinely aged out
rabbitmqadmin get queue=orders.expired.parked count=10

If the price queue drains to zero during the simulated outage, that’s correct — stale prices should expire. If the orders queue loses anything during the same test, the TTL strategy is wrong and you caught it on staging instead of during a rollout.

Where AI overreaches

It sometimes suggests TTL as a substitute for a max-length policy to control queue size. They’re not the same — TTL expires by age, max-length by count, and using TTL to bound size means your size limit drifts with consumer speed. Push back: “Do I actually want age-based expiry here, or count-based bounding? They behave differently under a slow consumer.” Often the right answer is a max-length policy, not a TTL.

It can also forget queue-level x-expires, which deletes the whole queue after a period of disuse — a great way to clean up ephemeral reply queues, and a great way to delete a queue you needed if you set it on the wrong one. When the AI proposes x-expires, confirm it means the queue, not the messages, and that the queue is genuinely disposable.

My loop

Tell the AI which messages are safe to drop and which aren’t, get a per-queue strategy that dead-letters anything important instead of deleting it, then run the slow-consumer test on staging to confirm only the disposable messages expire. The AI handles the fiddly per-message-versus-per-queue and head-of-line rules; the staging test confirms my deploy won’t quietly eat the orders.

These TTL-design prompts live with my other prompts, and the RabbitMQ category covers the dead-letter and max-length patterns that a sound expiration strategy leans on.