Redis Data Structure Selection Prompt

You are a senior SRE and Redis expert who models data structures for latency-sensitive systems.

I will provide:
- The use case and access patterns (reads, writes, queries)
- Approximate cardinality and value sizes
- Latency and memory constraints

Your job:

1. **Map the access pattern to a type**:
   - **String**: single blobs, counters (`INCR`), simple cache values, small JSON.
   - **Hash**: objects with independent fields — `HSET user:1 name Ada age 30`; fetch fields without deserializing the whole value.
   - **List**: ordered queues/stacks — `LPUSH`/`RPOP`, capped logs with `LTRIM`.
   - **Set**: unique membership, tags, dedup — `SADD`, `SISMEMBER`, set algebra (`SINTER`, `SUNION`).
   - **Sorted set (zset)**: ranked data, leaderboards, priority queues, time indexes — `ZADD`, `ZRANGEBYSCORE`.
   - **Stream**: append-only event log with consumer groups — `XADD`, `XREADGROUP`.
   - **Bitmap**: dense boolean flags per id — `SETBIT`, `BITCOUNT` (daily active users).
   - **HyperLogLog**: approximate unique counts at tiny memory — `PFADD`, `PFCOUNT` (~12KB, ~0.81% error).
2. **Consider cardinality and memory**: small hashes/zsets use compact listpack encoding; large ones convert to hashtable/skiplist.
3. **Avoid big keys**: a single multi-MB value or million-element collection blocks the event loop and hurts replication.
4. **Query needs**: range queries → zset; membership → set/bitmap; field access → hash.
5. **Atomicity**: pick types whose native commands give the atomic operation you need (`INCR`, `ZINCRBY`, `SADD`).
6. **TTL granularity**: TTL is per-key, not per-field — split by key if fields expire independently.
7. **Verify encoding**: `OBJECT ENCODING key` to confirm the compact form is in use.

Mark DESTRUCTIVE: `KEYS *` and `SMEMBERS`/`HGETALL`/`LRANGE 0 -1` on huge keys (event-loop blocking), `FLUSHALL`/`FLUSHDB`, and `DEBUG OBJECT` on prod.

---

Use case: [DESCRIBE]
Access patterns: [DESCRIBE]
Cardinality/value sizes: [DESCRIBE]

Why this prompt works

Redis performance is decided at the modeling stage. Choosing a hash over N strings, a zset over an app-side sort, or a HyperLogLog over a giant set changes memory and latency by orders of magnitude. This prompt matches each access pattern to the native type whose commands give you the operation atomically, and flags the big-key and encoding traps that only surface in production.

How to use it

1. Describe the query you need most — range, membership, ranking, or field access.
2. Give real cardinality — 100 elements and 10 million elements point to different types.
3. State value sizes so big-key risk can be assessed.
4. Note atomicity needs — counters and ranks have native atomic ops.

Useful commands

# Confirm the compact encoding is in use
redis-cli OBJECT ENCODING user:1        # listpack, intset, embstr, skiplist...

# Type and size checks
redis-cli TYPE leaderboard
redis-cli MEMORY USAGE leaderboard
redis-cli ZCARD leaderboard
redis-cli SCAN 0 TYPE hash COUNT 100

# Approximate unique counts (tiny memory)
redis-cli PFADD daily:2026-07-03 user1 user2 user3
redis-cli PFCOUNT daily:2026-07-03

# Daily-active bitmap
redis-cli SETBIT dau:2026-07-03 123 1
redis-cli BITCOUNT dau:2026-07-03

Example config

# redis.conf — thresholds where compact encodings convert to full structures
hash-max-listpack-entries 128
hash-max-listpack-value 64
list-max-listpack-size 128
set-max-intset-entries 512
set-max-listpack-entries 128
zset-max-listpack-entries 128
zset-max-listpack-value 64

Common findings this catches

• Object stored as JSON string → can’t update one field cheaply; use a hash.
• App-side sorting → replace with a sorted set and ZRANGE.
• Giant set for counting → swap to HyperLogLog for a huge memory win.
• List used for membership tests → O(N) LPOS; use a set for O(1).
• Big keys → collection outgrew a sane size; shard by sub-key.
• Encoding downgrade → collection crossed a *-max-listpack-* threshold.

When to escalate

• Data model spans many key types with cross-key transactions — design review.
• Big-key hotspots causing latency — needs sharding or restructuring.
• Approximate vs exact tradeoffs with compliance impact — involve the data owner.