Redis Cluster Sharding Design Prompt

You are a senior SRE and Redis expert who designs and operates Redis Cluster deployments.

I will provide:
- Dataset size, throughput, and node count
- Key access patterns (multi-key ops? transactions?)
- Current `CLUSTER INFO`/`CLUSTER NODES` if it exists

Your job:

1. **Slot model**: Cluster shards the keyspace into 16384 hash slots; slot = `CRC16(key) mod 16384`. Each primary owns a slot range; replicas back each primary.
2. **Node sizing**: at least 3 primaries (3 masters + 3 replicas is the minimum production shape) so failover has a majority. Distribute slots evenly.
3. **Multi-key constraint**: `MGET`, `MSET`, transactions, and Lua touching multiple keys work ONLY if all keys map to the same slot — else `CROSSSLOT` error.
4. **Hash tags**: force related keys into the same slot with `{...}` — only the substring inside braces is hashed, e.g. `user:{123}:profile` and `user:{123}:cart` co-locate. Use sparingly to avoid hot slots.
5. **Resharding**: move slots with `redis-cli --cluster reshard` or `CLUSTER SETSLOT ... MIGRATING/IMPORTING`; live migration streams keys and redirects with `ASK`/`MOVED`.
6. **Client requirements**: clients must be cluster-aware to follow `MOVED`/`ASK` redirects and cache the slot map.
7. **Failover**: replicas auto-promote on primary failure if a majority of primaries agree; `cluster-node-timeout` tunes detection.
8. **Validate balance**: `CLUSTER SLOTS`, `--cluster check`, and per-node key counts to catch skew and hot slots.

Mark DESTRUCTIVE: `CLUSTER RESET HARD` (wipes node's cluster state), `CLUSTER FORGET`/`CLUSTER SETSLOT` mistakes (orphan/lost slots), `FLUSHALL` across the cluster, and `KEYS *`/`DEBUG` on prod nodes.

---

Dataset/throughput/nodes: [DESCRIBE]
Access patterns: [DESCRIBE]
CLUSTER NODES: [PASTE]

Why this prompt works

Redis Cluster gives horizontal scale but silently changes the rules: multi-key operations only work within a slot, clients must follow redirects, and one careless hash tag creates a hot shard. This prompt grounds the design in the 16384-slot model, shows exactly how hash tags co-locate related keys, and walks live resharding with MIGRATING/IMPORTING so you don’t orphan slots.

How to use it

1. List your multi-key operations — they dictate hash-tag placement.
2. Give node count and dataset size — drives slot distribution and shard sizing.
3. Paste CLUSTER NODES/CLUSTER INFO for the current topology.
4. Confirm the client library is cluster-aware before going live.

Useful commands

# Cluster health and topology
redis-cli -c CLUSTER INFO            # cluster_state:ok, slots_assigned:16384
redis-cli -c CLUSTER NODES
redis-cli -c CLUSTER SLOTS
redis-cli --cluster check 10.0.0.10:6379

# Which slot / node owns a key
redis-cli -c CLUSTER KEYSLOT 'user:{123}:cart'
redis-cli -c CLUSTER GETKEYSINSLOT 866 10

# Create and reshard a cluster
redis-cli --cluster create 10.0.0.10:6379 10.0.0.11:6379 10.0.0.12:6379 \
  10.0.0.13:6379 10.0.0.14:6379 10.0.0.15:6379 --cluster-replicas 1
redis-cli --cluster reshard 10.0.0.10:6379

# Add a node then rebalance slots
redis-cli --cluster add-node 10.0.0.16:6379 10.0.0.10:6379
redis-cli --cluster rebalance 10.0.0.10:6379

Example config

# redis.conf — cluster node
port 6379
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000
cluster-require-full-coverage yes
cluster-replica-no-failover no
appendonly yes

Common findings this catches

• CROSSSLOT errors → multi-key ops span slots; add hash tags.
• Hot slot → one hash tag over-concentrates keys.
• Slot imbalance → uneven distribution; rebalance.
• Non-cluster client → ignores MOVED/ASK, hits wrong node.
• < 3 primaries → failover can’t reach majority.
• cluster_state:fail → uncovered slots; check --cluster check.

When to escalate

• Migration from single-node to Cluster — needs a staged cutover plan.
• Cross-slot transactional requirements — may need app-side redesign.
• Persistent hot shards — revisit key modeling and hash-tag strategy.