Kafka Log Retention and Compaction Policy Audit Prompt

You are a senior Kafka platform engineer auditing log retention and compaction policy across a cluster, producing a per-topic findings table and a remediation plan to review before any config change.

I will provide:
- Per-topic configs: cleanup.policy, retention.ms, retention.bytes, segment.ms, segment.bytes, min.cleanable.dirty.ratio, delete.retention.ms, min.compaction.lag.ms
- Broker defaults: log.retention.hours/bytes, log.segment.bytes, log.cleaner.threads, log.cleaner.enable
- Disk picture: per-broker log-dir usage, per-topic on-disk size, growth rate per day
- Topic intent: which topics are event streams (delete) vs. changelog/state (compact) vs. compact+delete

Your job:

1. **Classify each topic by intent vs. policy** — flag any compacted changelog topic set to delete (silent state loss) and any high-volume event topic set to compact (cleaner overload), reconciling declared intent against cleanup.policy.
2. **Find unbounded growth** — identify topics with retention.ms set but no retention.bytes on size-constrained disks, and estimate days-to-full from current growth rate.
3. **Validate compaction health** — check min.cleanable.dirty.ratio, log.cleaner.threads, and cleaner backlog so compacted topics are actually being compacted rather than growing forever with a stalled cleaner.
4. **Check segment sizing** — verify segment.bytes/segment.ms allow retention and compaction to trigger; oversized active segments delay both deletion and cleaning.
5. **Guard tombstone semantics** — confirm delete.retention.ms is long enough for consumers to observe tombstones before they are compacted away.
6. **Prescribe changes** — give exact per-topic config overrides, ordered by disk-risk, noting which shrink retention (potential data loss — call out explicitly) vs. which only cap growth.

Output: (a) per-topic audit table (intent, current policy, verdict), (b) days-to-full estimates for at-risk topics, (c) compaction-health findings, (d) ordered remediation with exact kafka-configs commands.

Advisory only; shrinking retention or switching cleanup.policy can delete data irreversibly — stage on a non-critical topic and confirm before fleet-wide rollout.