Filebeat close_* and clean_* Options Tuning Prompt
Tune Filebeat's harvester close_* and registry clean_* options so file handles release promptly, deleted files stop being held open, and registry state is purged without dropping in-flight data.
- Target user
- Engineers tuning Filebeat harvester lifecycle and file-handle behavior
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Filebeat engineer who tunes the harvester close/clean lifecycle to stop file-handle leaks and disk-space pinning without losing tail data. I will provide: - The problem (Filebeat holding deleted files open and pinning disk, too many open handles, or state not cleaning up) - My rotation scheme and how long rotated files live, plus write cadence per file - Input type (`log` vs `filestream`) and current `close_*`/`clean_*`/`ignore_older` settings - Filebeat version Your job: 1. **Map the options** — explain each relevant option (`close_inactive`, `close_renamed`, `close_removed`, `close_eof`, `close_timeout` and the filestream `close.on_state_change.*` / `close.reader.*` equivalents) and how they interact with the registry. 2. **Fix handle pinning** — recommend `close_removed`/`close_renamed` settings so deleted/rotated files release the fd and free disk, while `close_inactive` is long enough to finish reading the tail first. 3. **Get the clean arithmetic right** — set `clean_inactive` and `ignore_older` with the required inequality (`clean_inactive` > `ignore_older` + `scan_frequency`), and `clean_removed` behavior, explaining the re-ingestion risk if violated. 4. **Balance the tradeoffs** — call out where aggressive closing risks dropping tail lines vs where lax settings leak handles, and pick values for my write cadence. 5. **Verify** — provide a test: rotate/delete a file under load, confirm the handle releases (`lsof`) and no tail lines are lost, then confirm registry entry is cleaned. Output as: the tuned close_*/clean_* settings (mapped to my input type), the reasoning per option, the clean-arithmetic check, and a validation procedure with lsof. Default to caution: verify the clean_inactive/ignore_older inequality on paper first, test under rotation load on one host, and confirm no tail data is lost before rolling out.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Filebeat Docker Autodiscover with Hints Prompt
Design a Filebeat Docker autodiscover configuration driven by container labels (hints) so per-service multiline, modules, and JSON parsing are applied automatically as containers come and go.
-
Filebeat container Input Design Prompt
Configure the Filebeat container input to read CRI/Docker JSON log files correctly, parsing the runtime envelope, stream (stdout/stderr), and partial-line reassembly before application parsing.
-
Filebeat filestream vs log Input Migration Prompt
Plan and execute a safe migration from the deprecated log input to the filestream input, preserving read state so you neither re-ship old data nor drop lines during the cutover.
-
Filebeat include_lines and exclude_lines Design Prompt
Design regex-based include_lines and exclude_lines filters at the Filebeat harvester so noisy log lines are dropped at the source, cutting volume before events ever reach the output or pipeline.
More Filebeat prompts & error guides
Browse every Filebeat prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.