Redis Object Encoding Optimization Prompt
Cut Redis memory by keeping values in their compact internal encodings — listpack, intset, embstr — by tuning the *-max-* thresholds and reshaping keys, verified with OBJECT ENCODING.
- Target user
- Engineers reducing Redis memory footprint
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior engineer who reduces Redis memory by keeping values in their compact internal encodings. I will provide: - `INFO memory` (used_memory, dataset size), a few representative keys, and their sizes - The access patterns on the biggest namespaces - Current `*-max-*` config thresholds Your job: 1. **Explain the encodings**: each type has a compact form for small values and a full form for large ones: - Hash/Set/List/ZSet → **listpack** (small) vs hashtable/skiplist/quicklist (large). - Set of integers → **intset** (very compact) until it exceeds `set-max-intset-entries` or gains a non-integer. - String → **int** (integers), **embstr** (≤44 bytes), **raw** (longer). 2. **Read the thresholds** and their effect: `hash-max-listpack-entries`/`-value`, `list-max-listpack-size`, `set-max-listpack-entries`/`-value`, `set-max-intset-entries`, `zset-max-listpack-entries`/`-value`. Crossing any threshold converts the whole key to the memory-heavier encoding — and Redis does **not** convert back automatically. 3. **Verify with `OBJECT ENCODING`**: for each hot key, show whether it is on the compact or heavy encoding and quantify the memory difference with `MEMORY USAGE`. 4. **Reshape to stay compact**: the biggest win is often splitting one huge hash into many small hashes (bucketing by a hash of the field) so each stays under the listpack threshold — the classic "hash-field sharding" trick that stores millions of small values densely. 5. **Tune thresholds deliberately**: raising a `*-max-listpack-*` limit keeps larger collections compact but makes operations on them O(N) (listpack is linear); balance memory vs CPU. Note new keys pick up new limits; existing keys keep their encoding until rewritten. 6. **Confirm the win**: compare `used_memory` and `MEMORY USAGE` before/after, and check `latency` didn't regress from larger listpacks. Mark DESTRUCTIVE or risky: raising listpack limits so high that O(N) ops stall the event loop, rewriting millions of keys on the primary without rate limiting, and assuming a threshold change re-compacts existing keys (it does not). --- INFO memory + sample keys: [PASTE] Access patterns: [DESCRIBE] Current thresholds: [PASTE]
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Why this prompt works
Redis stores small collections in compact “listpack”/“intset” encodings and silently upgrades to memory-heavy hashtables/skiplists once a size threshold is crossed — and never downgrades. This prompt makes you measure the encoding with OBJECT ENCODING, tune the *-max-* thresholds knowingly (memory vs O(N) CPU), and apply the hash-field-sharding trick that is usually the single biggest memory win.
How to use it
- Dump
OBJECT ENCODINGfor the biggest keys — you can’t optimize what you haven’t measured. - Identify one huge hash/set — splitting it into small buckets is often a multiplier-sized saving.
- Change one threshold at a time and re-measure memory and latency.
Useful commands
# Which encoding is a key actually using?
redis-cli OBJECT ENCODING user:42
redis-cli OBJECT ENCODING bigset
# Memory cost of the key
redis-cli MEMORY USAGE user:42
# Current encoding thresholds
redis-cli CONFIG GET 'hash-max-listpack-*'
redis-cli CONFIG GET 'set-max-*'
redis-cli CONFIG GET 'zset-max-listpack-*'
# Keep larger hashes compact (new keys), then verify
redis-cli CONFIG SET hash-max-listpack-entries 512
Example: hash-field sharding
# Instead of one giant hash 'counters' with 10M fields (hashtable, heavy):
# bucket the field so each shard stays under the listpack threshold
key = "counters:" + (crc32(field) % 4096)
field = field
# each 'counters:<bucket>' holds ~2.4k fields → stays listpack → far denser
Common findings this catches
- One giant hash on hashtable encoding → shard it into listpack buckets.
- Intset upgraded by a stray string member → keep sets integer-only.
- Thresholds left at defaults → collections tip to heavy encoding early.
- Threshold raised too high → O(N) commands stall the loop.
- Expecting auto-recompaction → must rewrite keys to re-encode.
When to escalate
- The dataset is genuinely too big for one node even when compact — plan clustering.
- Memory vs latency tradeoff is unclear — benchmark listpack sizes under real command mix.
Related prompts
-
Redis Memory Optimization Prompt
Analyze Redis memory usage — encodings, big keys, fragmentation — and reduce footprint with listpack/intset thresholds and smarter modeling.
-
Redis Bitmap Analytics Design Prompt
Design memory-efficient real-time analytics with Redis bitmaps — daily active users, retention, and feature flags — using SETBIT/BITCOUNT/BITOP and BITFIELD counters.
-
Redis HyperLogLog Cardinality Design Prompt
Design HyperLogLog counting on Redis — PFADD/PFCOUNT/PFMERGE, error bounds, key rollups, and Cluster merges — for unique-count analytics.
-
Redis TTL and Expiration Strategy Prompt
Design TTL hygiene with EXPIRE/PEXPIRE, understand active vs lazy expiry, and avoid immortal keys and expiry-driven latency spikes.
More Redis prompts & error guides
Browse every Redis prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.