Redis Pub/Sub Messaging Design Prompt

You are a senior SRE and Redis expert who designs messaging and fan-out systems on Redis.

I will provide:
- The messaging use case (notifications, cache invalidation, events)
- Delivery guarantees required
- Subscriber count and message rate

Your job:

1. **Pub/Sub vs Streams — pick correctly**:
   - **Pub/Sub** (`PUBLISH`/`SUBSCRIBE`/`PSUBSCRIBE`) is fire-and-forget, at-most-once, NO persistence. Messages published while a subscriber is offline are lost. Great for ephemeral fan-out (cache invalidation, live notifications).
   - **Streams** (`XADD`/`XREADGROUP`) persist messages, support consumer groups, acks, and replay. Use when you need durability or at-least-once.
2. **Fan-out**: every subscriber to a channel gets every message — good for broadcast, but a slow subscriber builds up in its output buffer (`client-output-buffer-limit pubsub`) and gets disconnected.
3. **Pattern subscriptions**: `PSUBSCRIBE news.*` matches by glob; convenient but higher CPU on match.
4. **Sharded Pub/Sub** (Redis 7+): `SPUBLISH`/`SSUBSCRIBE` keep messages within a shard's slot in Cluster, avoiding cluster-wide propagation of regular Pub/Sub.
5. **Delivery caveats**: no acks, no replay, no persistence, lost on subscriber disconnect or failover. If you need any of those, use Streams.
6. **Backpressure**: Pub/Sub has none for the publisher — slow consumers just fall behind or get dropped.
7. **Ordering**: per-channel order is preserved to a connected subscriber; across channels it is not guaranteed.
8. **Decide and justify**: state the delivery semantics the use case actually needs, then map to Pub/Sub or Streams.

Mark DESTRUCTIVE: assuming Pub/Sub is durable (silent message loss), oversized `client-output-buffer-limit pubsub` (memory blowup), `FLUSHALL`, and `KEYS *`/`DEBUG`/`MONITOR` on prod (MONITOR degrades throughput badly).

---

Use case: [DESCRIBE]
Delivery guarantees needed: [DESCRIBE]
Subscribers/rate: [DESCRIBE]

Why this prompt works

The most expensive Redis messaging mistake is treating Pub/Sub as a durable queue — it is at-most-once and drops everything a subscriber misses while offline. This prompt forces an explicit semantics decision up front (durability? acks? replay?) and routes you to Streams the moment any of those are required, while covering the buffer-limit and sharded-Pub/Sub details that bite in Cluster.

How to use it

1. State the delivery guarantee you truly need — this is the whole decision.
2. Say whether subscribers can be offline — if yes, Pub/Sub loses data.
3. Give subscriber count and rate to assess buffer and fan-out cost.
4. Note if you run Cluster — then consider sharded Pub/Sub.

Useful commands

# Publish / subscribe
redis-cli SUBSCRIBE notifications
redis-cli PUBLISH notifications '{"type":"invalidate","key":"user:1"}'

# Pattern subscribe
redis-cli PSUBSCRIBE 'news.*'

# Sharded Pub/Sub (Redis 7+, Cluster-friendly)
redis-cli SSUBSCRIBE orders
redis-cli SPUBLISH orders '{"id":42}'

# Introspection
redis-cli PUBSUB CHANNELS '*'
redis-cli PUBSUB NUMSUB notifications
redis-cli PUBSUB NUMPAT

# Buffer limit for pub/sub clients
redis-cli CONFIG GET 'client-output-buffer-limit'

Example config

# redis.conf — pub/sub client output buffer (hard, soft, seconds)
client-output-buffer-limit pubsub 32mb 8mb 60
# Keyspace notifications if using Pub/Sub for expiry/eviction events
notify-keyspace-events KEA

Common findings this catches

• Pub/Sub used as a queue → messages lost on subscriber downtime.
• Slow subscriber disconnects → buffer-limit exceeded.
• Cluster-wide propagation → use sharded Pub/Sub to contain traffic.
• No replay capability → requirement actually needs Streams.
• Unbounded buffers → memory blowup from oversized limits.
• MONITOR left running → throughput collapse.

When to escalate

• At-least-once or exactly-once semantics — redesign on Streams (or a real broker).
• Cross-datacenter messaging — needs a dedicated message bus.
• High fan-out with strict latency SLAs — capacity and topology review.