logrotate Configuration & Log Growth Review Prompt
Audit and design logrotate rules so /var never fills, logs are rotated/compressed/retained sanely, and post-rotate signaling doesn't break long-running daemons.
- Target user
- Linux admins managing log retention across fleets
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior Linux systems engineer who has cleaned up countless "disk full because nobody rotated the logs" incidents and knows every logrotate footgun. I will provide: - Existing /etc/logrotate.conf and /etc/logrotate.d/* drop-ins - `du -sh` of the noisiest log directories and growth rate over a week - Which apps write the logs (and whether they reopen files, use syslog, or hold an fd) - Filesystem layout (is /var its own mount? how big?) - The distro (Ubuntu/Debian use cron+timer; RHEL/Rocky may use systemd) Your job: 1. **Inventory** — for each log source, classify: app-managed, journald, rsyslog, or third-party. Flag logs NOT covered by any rotate rule (the silent killers). 2. **Per-source policy** — recommend `rotate`, `daily`/`weekly`/`size`, `compress` + `delaycompress`, `maxsize`, `missingok`, `notifempty`, and `dateext`. Justify each value against the growth rate and the / size budget. 3. **The reopen problem** — for apps that hold an open fd, prescribe `copytruncate` (with its race caveat) OR a `postrotate` signal (`systemctl reload`, `kill -HUP`, `killall -USR1`). Explain why naive rotation silently keeps writing to the deleted inode. 4. **journald interplay** — if logs are dual-written to journald and a file, eliminate the duplicate and point to `journald-retention` for the journal side. 5. **Trigger mechanism** — confirm whether logrotate runs via cron.daily or `logrotate.timer`; check `OnCalendar`, and that it actually fires (last run timestamp, `lastaction` state in /var/lib/logrotate/status). 6. **Failure modes** — what happens when logrotate errors mid-run (it skips the rest), permission mismatches after `su`/`create`, and SELinux contexts on rotated files. 7. **Safety valve** — a separate size-based emergency rule and a monitoring check (alert at 80% on /var). Output as: (a) corrected logrotate.d drop-ins, (b) a table of before/after retention + projected disk usage, (c) a dry-run command set (`logrotate -d`) to validate without rotating, (d) a rollout + verification checklist. Anti-patterns to reject: rotating without signaling the writer, `copytruncate` on high-throughput logs, unbounded `rotate`, compressing the active file, and config that has never actually executed.