Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Filebeat Difficulty: Advanced ClaudeChatGPTCursor

Filebeat Conditional Output Routing Design Prompt

Route events from a single Filebeat to different Elasticsearch data streams, indices, or ingest pipelines based on conditions, so each log type lands in the right place with the right retention without running multiple agents.

Target user
Engineers routing mixed log streams to per-type indices, pipelines, or data streams in Filebeat
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior Filebeat engineer who splits one agent's mixed event stream to the correct destinations — data streams, indices, or ingest pipelines — using conditions, cleanly and without index explosion.

I will provide:
- The distinct log types this Filebeat handles and a few sample events for each (with the fields that distinguish them)
- Where each type should land (target data stream / index name, retention/ILM expectation, and any per-type ingest pipeline)
- My current `output.elasticsearch` block and whether I use classic indices or data streams
- Filebeat version

Your job:

1. **Choose the routing model** — decide between `output.elasticsearch.indices` with `when` conditions, `pipelines` with `when`, or per-type `data_stream` naming (dataset/namespace), and justify the choice for classic indices vs data streams.
2. **Write reliable conditions** — express each rule with the right condition (`equals`, `contains`, `regexp`, `has_fields`) against fields that actually exist at output time, and order them so the first match wins where that matters.
3. **Set a safe default** — define the fallback index/pipeline for anything that matches no rule, and make unmatched events observable (a distinct default or a tag) rather than silently blended in.
4. **Bound the destinations** — if a routing key comes from event data, constrain it to a known allow-list so a malformed or hostile value cannot spawn arbitrary indices; flag any unbounded key as a risk.
5. **Line up the backing config** — list the index templates, ILM policies, and pipelines each destination requires so routing does not fail on a missing template or an unknown pipeline.
6. **Verify** — give a test plan: feed one sample of each type, confirm it lands in the intended destination and pipeline, and confirm the default catches an unmatched sample.

Output as: the complete `output.elasticsearch` routing block, each condition explained in plain English, the list of templates/ILM/pipelines to create, and a per-type validation checklist.

Default to caution: always define and monitor a default destination, verify conditions against real events, and never let a user-controlled field become an unbounded index name.

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Related prompts

More Filebeat prompts & error guides

Browse every Filebeat prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.