Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Redis Difficulty: Advanced ClaudeChatGPTCursor

Redis HyperLogLog Cardinality Design Prompt

Design HyperLogLog counting on Redis — PFADD/PFCOUNT/PFMERGE, error bounds, key rollups, and Cluster merges — for unique-count analytics.

Target user
Senior engineers building unique-count analytics on Redis
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior SRE and Redis data-modeling expert who designs approximate unique-count ("cardinality") analytics on Redis in production.

I will provide:
- What is being counted (unique users, IPs, devices) and the time grain (per minute/hour/day)
- Expected cardinality per bucket and number of buckets
- Read patterns (single bucket, rolling window, merged totals)
- Topology (standalone / Sentinel / Cluster) and accuracy tolerance

Your job:

1. **Confirm HyperLogLog fits.** HLL answers "how many *distinct* items?" with ~0.81% standard error using a fixed **~12 KB** per key regardless of cardinality (up to billions). It cannot list members, test membership, or give exact counts. If the user needs any of those, HLL is the wrong tool — say so and suggest a set or Bloom filter.
2. **Design the key grain.** One HLL key per (metric, time-bucket): e.g. `uv:2026-07-08:homepage`. Smaller grains (per-minute) cost more keys but enable flexible rollups; larger grains (per-day) cost fewer keys but less resolution. Recommend a grain and a key-naming scheme.
3. **Plan rollups with PFMERGE.** Daily = `PFMERGE` of 24 hourly keys; weekly = merge of 7 daily. Merges are unioned correctly (no double counting), which set-based counting cannot match cheaply. Note `PFCOUNT key1 key2 ...` gives the union count without materializing a merge key.
4. **Handle Redis Cluster.** `PFMERGE` and multi-key `PFCOUNT` require all source and destination keys in the **same slot** — otherwise CROSSSLOT. Use a **hash tag** so a rollup family co-locates, e.g. `uv:{homepage}:2026-07-08:14`. Design the tag around the merge boundary.
5. **Set expiry.** Add `EXPIRE`/`EXPIREAT` per bucket so old fine-grained keys drop after they have been rolled up. Keep long-lived aggregate keys only.
6. **Understand the error.** 0.81% relative standard error means a true 1,000,000 might read 990k–1,010k. For dashboards that is fine; for billing it is not. State the tolerance explicitly.
7. **Write path.** `PFADD` is O(1) amortized and idempotent for repeated items — safe to call on every event. It returns 1 only when the estimate likely changed, which you can ignore.

Deliverables: key schema with hash-tag strategy, `PFADD`/`PFCOUNT`/`PFMERGE` templates, rollup + expiry plan, a memory estimate (keys × ~12 KB), and the accuracy statement.

Mark DESTRUCTIVE: `PFMERGE` that overwrites an existing aggregate key, `FLUSHDB`, and replacing an HLL key with a `SET` "for exactness" (blows up memory from 12 KB to unbounded).

---

What/grain: [DESCRIBE]
Cardinality/buckets: [DESCRIBE]
Topology/tolerance: [DESCRIBE]

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Why this prompt works

The whole appeal of HyperLogLog is counting billions of distinct items in a fixed ~12 KB — but that only holds if the key grain, the PFMERGE rollup plan, and the Cluster hash-tag layout are designed together. Teams that skip this end up with CROSSSLOT errors on merges, thousands of un-expired buckets, or someone quietly swapping in a giant SET for “accuracy.” This prompt pins down the grain, the merge boundary, the slot strategy, and the error budget before any code ships.

How to use it

  1. Name the metric and time grain so the key scheme is concrete.
  2. Estimate cardinality and bucket count for a memory number.
  3. Describe the rollups you need (hourly→daily→weekly) to design PFMERGE.
  4. State the accuracy tolerance — this decides whether HLL is even valid.

Useful commands

# Count uniques into an hourly bucket (idempotent per item)
redis-cli PFADD uv:{homepage}:2026-07-08:14 user:42 user:99

# Estimate distinct count for one bucket
redis-cli PFCOUNT uv:{homepage}:2026-07-08:14

# Union count across hours WITHOUT materializing a key
redis-cli PFCOUNT uv:{homepage}:2026-07-08:13 uv:{homepage}:2026-07-08:14

# Roll 24 hourly keys into a daily aggregate (same slot via hash tag)
redis-cli PFMERGE uv:{homepage}:2026-07-08 \
  uv:{homepage}:2026-07-08:00 uv:{homepage}:2026-07-08:01  # ...through :23

# Expire fine-grained buckets after rollup
redis-cli EXPIRE uv:{homepage}:2026-07-08:14 172800   # 48h

# Inspect: an HLL is a special string, ~12 KB dense
redis-cli MEMORY USAGE uv:{homepage}:2026-07-08
redis-cli TYPE uv:{homepage}:2026-07-08                 # string

Cluster note

# WRONG: keys land in different slots -> CROSSSLOT
PFMERGE uv:day  uv:hour:00  uv:hour:01

# RIGHT: hash tag {homepage} forces one slot for the whole family
PFMERGE uv:{homepage}:day  uv:{homepage}:00  uv:{homepage}:01

Common findings this catches

  • Exact-count requirement → HLL is invalid; use a set or exact counter.
  • CROSSSLOT on PFMERGE → missing hash tag around the rollup family.
  • Thousands of un-expired buckets → add TTLs, keep only aggregates.
  • SET-based unique counting → gigabytes where 12 KB would do.
  • Membership/listing needed → wrong primitive; HLL can’t do it.

When to escalate

  • Sub-1% error is unacceptable — move to exact structures and accept the memory cost.
  • Very high bucket counts strain key space — consolidate grain or shard.
  • Need both count and membership — pair an HLL with a Bloom filter or set.

Related prompts

More Redis prompts & error guides

Browse every Redis prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.