Kafka Consumer Offset Reset and Replay Plan Prompt
Plan a safe consumer-group offset reset or message replay — by timestamp, offset, or shift — without double-processing, skipping data, or corrupting downstream state.
- Target user
- Backend and data engineers
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Kafka engineer planning an offset reset / replay for a consumer group after a bad deploy, a poison message, or a downstream data-loss incident. Produce a precise, reversible plan before touching committed offsets. I will provide: - The scenario: reprocess a time window, skip a poison offset, rewind after a downstream bug, or seek to latest to drop a backlog - Group id, affected topic(s) and partition count, and whether consumers can be stopped - Downstream behavior: is processing idempotent, exactly-once (transactional), or at-least-once with side effects (emails, payments, writes)? - The target: a timestamp, absolute offsets, "shift by N", earliest, or latest Your job: 1. **Require consumers stopped first** — insist the group be fully stopped (no active members) before resetting, because kafka-consumer-groups.sh --reset-offsets refuses to run on a live group, and explain why. 2. **Choose the right reset spec** — map the scenario to the correct flag: --to-datetime for time-window replay, --to-offset / --shift-by for surgical moves, --to-earliest / --to-latest for full rewind or backlog drop; scope it with --topic or --all-topics and per-partition where needed. 3. **Mandate a dry run** — always run with --dry-run first to print the proposed new offsets, review them against the intended window, and only then re-run with --execute. 4. **Assess reprocessing side effects** — determine whether replay will double-fire non-idempotent side effects; if so, recommend guards (idempotency keys, dedup, disabling side effects during replay) or a shadow group. 5. **Plan reversibility** — capture current committed offsets before the reset so you can restore them, and define how to verify lag and correctness after resuming. Output: (a) the pre-reset checklist (stop group, snapshot current offsets), (b) the exact kafka-consumer-groups.sh reset command with the right spec, shown as --dry-run then --execute, (c) side-effect/idempotency safeguards, (d) the rollback offsets, (e) post-resume verification steps. Advisory only; capture the existing offsets and dry-run every reset before executing, and prefer a separate replay group over rewinding a production group when side effects aren't idempotent.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Kafka Consumer Rebalance Storm Triage Prompt
Diagnose frequent or looping consumer-group rebalances by working through session, heartbeat, and poll timeouts, static membership, and the rebalance protocol in use.
-
Kafka Connect Pipeline Debugging & Tuning Prompt
Diagnose a failing or lagging Kafka Connect pipeline — dead or restarting tasks, DLQ growth, offset stalls, and rebalance churn — then prescribe connector, converter, and worker tuning that is safe to roll out.
-
Kafka Consumer Lag Investigation Prompt
Investigate and reduce growing consumer lag by isolating the root cause — slow processing, partition skew, GC pauses, or broker-side bottlenecks — then prescribe targeted fixes.
-
Kafka Client Quota and Throttling Design Prompt
Design produce, fetch, and request-percentage quotas per user/client-id so one noisy tenant cannot saturate broker network or CPU and starve others on a shared cluster.
More Kafka prompts & error guides
Browse every Kafka prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.