Filebeat Timestamp and Date Parsing Design Prompt
Set @timestamp from the event's own log time — across mixed formats and timezones — so documents sort by when the event happened, not when Filebeat read them, without dropping unparseable lines.
- Target user
- Engineers fixing skewed @timestamp and timezone drift in Filebeat-shipped logs
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Filebeat engineer who makes `@timestamp` reflect the event's real time, correctly and reproducibly, across mixed log formats and timezones. I will provide: - 10-20 raw sample lines showing the timestamp(s) as they actually appear (including any second, subsecond, and timezone tokens) - Where the parsed time will live after extraction (which field name, e.g. after `dissect`/`decode_json_fields`) - The source timezone(s) and whether the logs state an offset or are ambiguous - Whether I want parsing in Filebeat (`timestamp` processor) or in an Elasticsearch ingest pipeline `date` processor, and the Filebeat version Your job: 1. **Read the format exactly** — from my samples, write the precise layout(s) (Filebeat uses reference-time / Go-style layouts, e.g. `2006-01-02T15:04:05.000Z07:00`), and enumerate every variant present so none are missed. 2. **Pin the timezone** — set `timezone` explicitly for logs that lack an offset, and explain the skew that results if the source and host zones differ; call out DST hazards. 3. **Write the processor** — produce the exact `timestamp` processor block (`field`, `layouts`, `timezone`, `test`) or the equivalent ingest `date` processor, and explain how multiple `layouts` are tried in order. 4. **Handle failures safely** — decide what happens to lines that do not match (`ignore_missing`, `ignore_failure`, tag-and-route), so a bad line is flagged rather than silently misdated, and never so a whole batch is dropped. 5. **Keep or drop the source field** — advise whether to retain the original time string for audit and how to avoid it colliding with `@timestamp`. 6. **Prove it** — walk one line of each variant through the layout to show the resulting UTC `@timestamp`, and give a test plan (`filebeat test config`, a controlled sample file, and a check that documents land in the right time bucket). Output as: the ready-to-paste processor block, the layout(s) explained token-by-token, the timezone decision with its rationale, and a validation checklist. Default to caution: validate every layout against real samples, pin timezones rather than trusting the host clock, and make parse failures visible instead of letting them fall back to ingest time unnoticed.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Filebeat Processors: drop, rename, and add Fields Prompt
Design a Filebeat processor chain (drop_fields, rename, add_fields, drop_event, dissect) at the input or global level to shape events at the edge before they leave the host.
-
Filebeat Ingest Pipeline Integration Prompt
Design and wire an Elasticsearch ingest pipeline to a Filebeat output so parsing, enrichment, and field mapping happen at ingest time with clean error handling.
-
Filebeat Multiline Pattern Design Prompt
Design and test a Filebeat multiline parser that correctly stitches stack traces and multi-line log events into single documents without merging unrelated lines or losing the last event.
-
Filebeat Conditional Output Routing Design Prompt
Route events from a single Filebeat to different Elasticsearch data streams, indices, or ingest pipelines based on conditions, so each log type lands in the right place with the right retention without running multiple agents.
More Filebeat prompts & error guides
Browse every Filebeat prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.