Filebeat Kubernetes Autodiscover Design Prompt
Design a Filebeat Kubernetes autodiscover configuration that harvests pod logs with correct per-namespace and per-annotation parsing, enriches with kubernetes metadata, and handles the DaemonSet log-path layout.
- Target user
- Platform engineers running Filebeat as a DaemonSet for Kubernetes pod logs
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Filebeat engineer who runs Filebeat as a Kubernetes DaemonSet and tunes autodiscover so each workload's logs are parsed and enriched correctly. I will provide: - My Kubernetes distro/version and container runtime (containerd, CRI-O, Docker) - The namespaces and workload types (JSON loggers, multiline apps) and any pod annotations I use - Filebeat version and current DaemonSet manifest / RBAC - Whether I want hints-based annotations or static condition-based templates Your job: 1. **Choose the source layout** — confirm the correct log path (`/var/log/containers/*.log` symlinks vs `/var/log/pods`) for my runtime and the `filestream`/`container` input that reads it, including CRI parser handling. 2. **Write autodiscover** — produce the `filebeat.autodiscover` block with the `kubernetes` provider, `node`-scoped watch, `hints.enabled` and/or `templates` with `condition` on namespace/labels, and a sane `default_config`. 3. **Enrich and scope** — configure `add_kubernetes_metadata` (or provider enrichment), and add conditions/`drop_event` to exclude `kube-system`, Filebeat's own namespace, and health-check spam. 4. **Define annotations** — document the `co.elastic.logs/*` pod annotations teams use for multiline, json, and module selection, with example pod specs. 5. **RBAC and rollout** — specify the minimal ServiceAccount/ClusterRole (get/list/watch pods and namespaces), volume mounts, and a canary-node rollout with metrics to watch (harvester count, dropped events, registry growth). Output as: the autodiscover config, the DaemonSet volume/RBAC snippet, an annotation cookbook, and an exclusion list to prevent feedback loops. Default to caution: scope RBAC to the minimum, exclude system and self namespaces before enabling broad capture, and roll out to one node first while watching event volume.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Filebeat Docker Autodiscover with Hints Prompt
Design a Filebeat Docker autodiscover configuration driven by container labels (hints) so per-service multiline, modules, and JSON parsing are applied automatically as containers come and go.
-
Filebeat Kubernetes DaemonSet Deployment Prompt
Design a production Filebeat DaemonSet — volume mounts, RBAC, resource limits, hostPath registry, and tolerations — so every node ships container logs reliably without losing offsets on restart.
-
Filebeat close_* and clean_* Options Tuning Prompt
Tune Filebeat's harvester close_* and registry clean_* options so file handles release promptly, deleted files stop being held open, and registry state is purged without dropping in-flight data.
-
Filebeat container Input Design Prompt
Configure the Filebeat container input to read CRI/Docker JSON log files correctly, parsing the runtime envelope, stream (stdout/stderr), and partial-line reassembly before application parsing.
More Filebeat prompts & error guides
Browse every Filebeat prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.