Redis SCAN-Based Key Migration Design Prompt
Design a non-blocking key migration or bulk operation on a live Redis instance using SCAN cursors, batching, and rate limiting — never KEYS — with DUMP/RESTORE or MIGRATE.
- Target user
- SREs migrating or bulk-editing live Redis keys
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior SRE who designs safe, non-blocking bulk operations over a live Redis keyspace. I will provide: - The operation (rename a namespace, add TTLs, re-encode values, move keys to another instance/cluster) - The approximate key count and instance QPS/latency budget - Source and destination topology (same instance, another instance, standalone → cluster) Your job: 1. **Ban `KEYS`**: `KEYS pattern` is O(N) and blocks the single thread — on millions of keys it stalls the whole instance. Everything below uses `SCAN`. 2. **Design the SCAN loop**: iterate the cursor until it returns `0`, using `MATCH` to filter and `COUNT` as a hint (default 10; raise to 100–1000 to reduce round trips). Warn that `SCAN` gives weak guarantees — keys present for the whole scan are returned at least once, but keys added/removed mid-scan may be missed or duplicated, and the same key may appear twice. The operation must be idempotent. 3. **Use TYPE-aware SCANs where possible**: `HSCAN`/`SSCAN`/`ZSCAN` for large collections so you don't pull a whole big key at once. 4. **Batch and rate-limit**: process N keys per iteration, pipeline the writes, and sleep between batches to cap the added load; watch latency and back off. 5. **Pick the move mechanism**: - Same instance: `RENAME`/re-`SET` under the new key, then delete the old (idempotent, dual-read window). - Across instances: `DUMP` + `RESTORE` (add `REPLACE`), or `MIGRATE` (atomic move, `COPY` to keep source, `REPLACE` to overwrite) — mind that `MIGRATE` blocks both ends for the keys moved. - Whole datasets: prefer replication/`redis-cli --cluster` tooling over a hand-rolled scan. 6. **Preserve TTLs**: `DUMP`/`RESTORE` keeps them if you pass the TTL; a naive `GET`+`SET` drops TTLs — restore them explicitly. 7. **Make it resumable + verified**: checkpoint the cursor, and verify counts/`TYPE`/`TTL` on the destination before deleting the source. Mark DESTRUCTIVE or risky: `KEYS`/`FLUSHALL` on prod, `MIGRATE` without understanding it blocks, `RESTORE` without `REPLACE` (BUSYKEY on retries), and deleting source keys before verifying the destination. --- Operation: [DESCRIBE] Key count + QPS budget: [DESCRIBE] Source → destination: [DESCRIBE]
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Why this prompt works
The number-one way engineers take down a Redis instance is running KEYS * on a live box. This prompt replaces that with a resumable SCAN loop, forces idempotency (because SCAN can repeat and miss keys), preserves TTLs through DUMP/RESTORE, and verifies the destination before anything is deleted.
How to use it
- State the key count and latency budget — they set
COUNTand the inter-batch sleep. - Confirm the operation is idempotent —
SCAN’s weak guarantees require it. - Say whether TTLs must survive — it decides
DUMP/RESTOREvs a plain re-SET.
Useful commands
# Non-blocking iteration with server-side MATCH and a COUNT hint
redis-cli --scan --pattern 'legacy:*' --count 500
# Move a single key across instances, preserving TTL, overwrite-safe
TTL=$(redis-cli -h SRC PTTL mykey)
redis-cli -h SRC DUMP mykey | \
redis-cli -h DST -x RESTORE mykey "$TTL" REPLACE
# Atomic cross-instance move (keeps source with COPY)
redis-cli MIGRATE dst-host 6379 mykey 0 5000 COPY REPLACE
# Verify before deleting source
redis-cli -h DST TYPE mykey
redis-cli -h DST PTTL mykey
Common findings this catches
KEYSin a migration script → blocks prod.- Non-idempotent step →
SCANrepeats cause double-processing. - Dropped TTLs → migrated keys never expire.
RESTOREwithoutREPLACE→ BUSYKEY on retry.- Source deleted before verification → silent data loss.
When to escalate
- The dataset is too large for a hand-rolled scan — use replication or
redis-cli --clustermigration tooling. - The move is standalone → cluster with reshaping — plan hash-tag placement first.
Related prompts
-
Redis Backup and Migration Plan Prompt
Plan Redis backups with BGSAVE/RDB, move keys with DUMP/RESTORE and MIGRATE, and sequence safe version upgrades and data migrations.
-
Redis Bitmap Analytics Design Prompt
Design memory-efficient real-time analytics with Redis bitmaps — daily active users, retention, and feature flags — using SETBIT/BITCOUNT/BITOP and BITFIELD counters.
-
Redis Cache Invalidation Strategy Design Prompt
Design a correct cache-invalidation strategy for Redis — write-through vs write-behind vs cache-aside, versioned keys, and event-driven busting — so stale reads never outlive the source of truth.
-
Redis Cache Stampede Prevention Design Prompt
Design defenses against cache stampede (dogpile/thundering herd) when a hot Redis key expires and many clients recompute it at once — locks, early recompute, and jittered TTLs.
More Redis prompts & error guides
Browse every Redis prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.