Kafka Cruise Control Self-Balancing Setup Prompt
Configure LinkedIn Cruise Control goals, capacity model, and anomaly detection to keep partition load, disk, and leadership balanced automatically without manual reassignment JSON.
- Target user
- SRE and platform engineers
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Kafka SRE configuring Cruise Control for a cluster, producing a goals-and-capacity configuration plus a rollout plan to review before enabling any self-healing. I will provide: - Cluster shape: broker count, heterogeneity (mixed instance/disk types), rack layout, and RF/min.insync.replicas - Load picture: per-broker CPU, network in/out, disk usage and skew, and known hot brokers - Cruise Control state: whether the metrics reporter is deployed and how long the metrics window is - Goals: balance disk/leadership/network, respect rack awareness, and how aggressive self-healing should be Your job: 1. **Set the capacity model** — define capacity.json per-broker CPU, network in/out, and disk so heterogeneous brokers are modeled correctly, since a wrong capacity model produces bad rebalance proposals. 2. **Order the goals** — sequence hard goals (RackAwareGoal, ReplicaCapacityGoal, DiskCapacityGoal) before soft goals (ReplicaDistributionGoal, LeaderReplicaDistributionGoal, NetworkInbound/OutboundGoal), explaining why hard goals must never be violated. 3. **Tune the metrics window** — set the sampling window and required-samples so proposals are based on representative steady-state load, not a transient spike. 4. **Throttle rebalance traffic** — set the replication throttle and concurrent-partition-movement limits so self-balancing does not saturate the network and hurt production traffic. 5. **Configure anomaly detection** — decide which anomalies (broker failure, disk failure, goal violation, metric anomaly) trigger auto self-healing vs. notify-only, and the detection/fix delays. 6. **Plan rollout** — start in proposal-only/dry-run mode, review generated reassignments, then enable self-healing one anomaly type at a time. Output: (a) capacity.json outline, (b) ordered hard/soft goal list, (c) metrics-window and throttle settings, (d) anomaly self-healing matrix, (e) staged rollout from dry-run to auto. Advisory only; run Cruise Control in dry-run and review proposals against production load before enabling self-healing, which will move partitions on its own.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Kafka Consumer Rebalance Storm Triage Prompt
Diagnose frequent or looping consumer-group rebalances by working through session, heartbeat, and poll timeouts, static membership, and the rebalance protocol in use.
-
Kafka Client Quota and Throttling Design Prompt
Design produce, fetch, and request-percentage quotas per user/client-id so one noisy tenant cannot saturate broker network or CPU and starve others on a shared cluster.
-
Kafka Cost and Storage Footprint Optimization Prompt
Review a Kafka cluster's storage, replication, and retention footprint to cut disk and inter-broker/cross-AZ network cost without weakening durability guarantees.
-
Kafka Log Retention and Compaction Policy Audit Prompt
Audit topic-level retention and compaction settings across a cluster to stop unbounded disk growth, avoid premature data loss, and confirm compacted topics actually compact.
More Kafka prompts & error guides
Browse every Kafka prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.