Filebeat Kubernetes DaemonSet Deployment Prompt
Design a production Filebeat DaemonSet — volume mounts, RBAC, resource limits, hostPath registry, and tolerations — so every node ships container logs reliably without losing offsets on restart.
- Target user
- Platform engineers deploying Filebeat as a Kubernetes DaemonSet
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Kubernetes platform engineer who has run Filebeat DaemonSets across large clusters. You know the failure modes cold: lost registry, missing RBAC for autodiscover, unmounted log paths, and resource starvation during log storms. I will provide: - My current DaemonSet manifest (or that I'm starting fresh) and Filebeat config/ConfigMap: [MANIFEST/CONFIG] - Cluster runtime and log layout (containerd/CRI-O, `/var/log/containers` symlinks to `/var/log/pods`): [RUNTIME] - Where logs go (ES, Logstash, Kafka) and whether Filebeat autodiscover is used: [OUTPUT + DISCOVERY] - Node shape, taints/tolerations, and whether control-plane nodes should be covered: [NODES] - The goal/symptom (duplicate logs after rollout, missing container logs, OOM kills, RBAC errors, wanting per-pod annotations): [GOAL] Your job: 1. **Get the volume mounts right.** Confirm the read-only mounts for `/var/log/containers`, `/var/log/pods`, and the container runtime's log dir, and explain why both the symlink dir and the real target must be mounted (symlinks point across filesystems). 2. **Persist the registry.** Mandate a hostPath mount for `/usr/share/filebeat/data` so file offsets survive pod restarts and rollouts; explain the duplicate-vs-loss consequence of getting this wrong. 3. **Scope RBAC minimally.** Provide the ServiceAccount + ClusterRole/Binding with only the get/list/watch verbs Filebeat autodiscover needs on pods/namespaces/nodes — never write verbs. 4. **Set resources and scheduling.** Recommend CPU/memory requests+limits sized to expected throughput and the internal queue, `tolerations` to cover tainted/control-plane nodes if desired, `priorityClassName`, and an `updateStrategy` that rolls safely. 5. **Wire autodiscover/hints** if used — the `hints.enabled` provider and per-pod annotations for multiline/module/exclude, plus how to exclude Filebeat's own logs to avoid loops. Output as: (a) a checklist mapping each failure mode (lost registry, missing symlink target, RBAC, OOM) to the manifest fix, (b) the corrected DaemonSet manifest (volumes, mounts, resources, tolerations) commented, (c) the RBAC objects, (d) the Filebeat ConfigMap snippet for autodiscover. Treat a missing registry hostPath as a blocking issue, not a nice-to-have.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Filebeat Kubernetes Autodiscover Design Prompt
Design a Filebeat Kubernetes autodiscover configuration that harvests pod logs with correct per-namespace and per-annotation parsing, enriches with kubernetes metadata, and handles the DaemonSet log-path layout.
-
Filebeat Conditional Output Routing Design Prompt
Route events from a single Filebeat to different Elasticsearch data streams, indices, or ingest pipelines based on conditions, so each log type lands in the right place with the right retention without running multiple agents.
-
Filebeat Filestream High-Volume Harvester Scaling Prompt
Scale the filestream input on high-volume hosts so thousands of active files are harvested promptly without exhausting file descriptors, stalling the scanner, or letting slow outputs starve the harvesters.
-
Filebeat Timestamp and Date Parsing Design Prompt
Set @timestamp from the event's own log time — across mixed formats and timezones — so documents sort by when the event happened, not when Filebeat read them, without dropping unparseable lines.
More Filebeat prompts & error guides
Browse every Filebeat prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.