Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for Filebeat Difficulty: Advanced ClaudeChatGPTCursor

Filebeat filestream vs log Input Migration Prompt

Plan and execute a safe migration from the deprecated log input to the filestream input, preserving read state so you neither re-ship old data nor drop lines during the cutover.

Target user
Observability and platform engineers modernizing Filebeat harvesting configs
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior Filebeat engineer who has migrated large fleets off the deprecated `log` input onto `filestream` without duplicating or dropping a single line.

I will provide:
- My current `filebeat.inputs` block(s) using `type: log`
- The paths harvested, and whether they rotate (copytruncate vs rename) and how fast
- My Filebeat version, deployment method (systemd, Docker, DaemonSet), and registry location
- Any use of `multiline`, `exclude_lines`, `include_lines`, `fields`, or `close_*`/`clean_*` options

Your job:

1. **Map every option** — produce a side-by-side table translating each `log` input setting to its `filestream` equivalent (`paths` → `id` + `paths`, `multiline.*` → `parsers: - multiline:`, `close_inactive` → `close.on_state_change.inactive`, `scan_frequency` → `prospector.scanner.check_interval`, etc.), flagging any option with no direct equivalent.
2. **Design the id strategy** — explain why every `filestream` input REQUIRES a unique, stable `id`, how the id keys the registry, and how a changed id causes full re-harvest; recommend an id naming scheme.
3. **Plan the state cutover** — describe how filestream's take-over mode or a clean canary approach lets you switch without re-shipping historical data, and exactly what to watch in the registry and output volume to confirm success.
4. **Handle rotation** — configure the scanner and fingerprint/native file identity so rotated files are tracked correctly under the new input.
5. **Stage the rollout** — give a canary → subset → fleet sequence with the specific metrics (harvester count, registry entries, events/sec, duplicate detection) that gate each stage.

Output as: the migrated `filestream` config, the option-mapping table, a step-by-step cutover runbook, and a rollback plan.

Default to caution: prove the migration on one host, compare shipped volume against baseline before and after, and keep the old config version-controlled for rollback.

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Related prompts

More Filebeat prompts & error guides

Browse every Filebeat prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.