Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Redis Difficulty: Advanced ClaudeChatGPTCursor

Redis Object Encoding Optimization Prompt

Cut Redis memory by keeping values in their compact internal encodings — listpack, intset, embstr — by tuning the *-max-* thresholds and reshaping keys, verified with OBJECT ENCODING.

Target user
Engineers reducing Redis memory footprint
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior engineer who reduces Redis memory by keeping values in their compact internal encodings.

I will provide:
- `INFO memory` (used_memory, dataset size), a few representative keys, and their sizes
- The access patterns on the biggest namespaces
- Current `*-max-*` config thresholds

Your job:

1. **Explain the encodings**: each type has a compact form for small values and a full form for large ones:
   - Hash/Set/List/ZSet → **listpack** (small) vs hashtable/skiplist/quicklist (large).
   - Set of integers → **intset** (very compact) until it exceeds `set-max-intset-entries` or gains a non-integer.
   - String → **int** (integers), **embstr** (≤44 bytes), **raw** (longer).
2. **Read the thresholds** and their effect: `hash-max-listpack-entries`/`-value`, `list-max-listpack-size`, `set-max-listpack-entries`/`-value`, `set-max-intset-entries`, `zset-max-listpack-entries`/`-value`. Crossing any threshold converts the whole key to the memory-heavier encoding — and Redis does **not** convert back automatically.
3. **Verify with `OBJECT ENCODING`**: for each hot key, show whether it is on the compact or heavy encoding and quantify the memory difference with `MEMORY USAGE`.
4. **Reshape to stay compact**: the biggest win is often splitting one huge hash into many small hashes (bucketing by a hash of the field) so each stays under the listpack threshold — the classic "hash-field sharding" trick that stores millions of small values densely.
5. **Tune thresholds deliberately**: raising a `*-max-listpack-*` limit keeps larger collections compact but makes operations on them O(N) (listpack is linear); balance memory vs CPU. Note new keys pick up new limits; existing keys keep their encoding until rewritten.
6. **Confirm the win**: compare `used_memory` and `MEMORY USAGE` before/after, and check `latency` didn't regress from larger listpacks.

Mark DESTRUCTIVE or risky: raising listpack limits so high that O(N) ops stall the event loop, rewriting millions of keys on the primary without rate limiting, and assuming a threshold change re-compacts existing keys (it does not).

---

INFO memory + sample keys: [PASTE]
Access patterns: [DESCRIBE]
Current thresholds: [PASTE]

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Why this prompt works

Redis stores small collections in compact “listpack”/“intset” encodings and silently upgrades to memory-heavy hashtables/skiplists once a size threshold is crossed — and never downgrades. This prompt makes you measure the encoding with OBJECT ENCODING, tune the *-max-* thresholds knowingly (memory vs O(N) CPU), and apply the hash-field-sharding trick that is usually the single biggest memory win.

How to use it

  1. Dump OBJECT ENCODING for the biggest keys — you can’t optimize what you haven’t measured.
  2. Identify one huge hash/set — splitting it into small buckets is often a multiplier-sized saving.
  3. Change one threshold at a time and re-measure memory and latency.

Useful commands

# Which encoding is a key actually using?
redis-cli OBJECT ENCODING user:42
redis-cli OBJECT ENCODING bigset

# Memory cost of the key
redis-cli MEMORY USAGE user:42

# Current encoding thresholds
redis-cli CONFIG GET 'hash-max-listpack-*'
redis-cli CONFIG GET 'set-max-*'
redis-cli CONFIG GET 'zset-max-listpack-*'

# Keep larger hashes compact (new keys), then verify
redis-cli CONFIG SET hash-max-listpack-entries 512

Example: hash-field sharding

# Instead of one giant hash 'counters' with 10M fields (hashtable, heavy):
#   bucket the field so each shard stays under the listpack threshold
key   = "counters:" + (crc32(field) % 4096)
field = field
# each 'counters:<bucket>' holds ~2.4k fields → stays listpack → far denser

Common findings this catches

  • One giant hash on hashtable encoding → shard it into listpack buckets.
  • Intset upgraded by a stray string member → keep sets integer-only.
  • Thresholds left at defaults → collections tip to heavy encoding early.
  • Threshold raised too high → O(N) commands stall the loop.
  • Expecting auto-recompaction → must rewrite keys to re-encode.

When to escalate

  • The dataset is genuinely too big for one node even when compact — plan clustering.
  • Memory vs latency tradeoff is unclear — benchmark listpack sizes under real command mix.

Related prompts

More Redis prompts & error guides

Browse every Redis prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.