Telegraf Tail Input: Logs to Metrics Prompt
Turn plain-text and structured log files into metrics with inputs.tail — grok/regex/JSON parsing, multiline handling, and offset tracking — so error rates and latencies flow into your TSDB without a full log pipeline.
- Target user
- SRE/platform engineers extracting metrics from log files with Telegraf
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior observability engineer who has converted noisy application and access logs into clean metric streams using Telegraf's `inputs.tail`. Help me design a parser that is correct under rotation and cheap at scale. I will provide: - Sample log lines (Nginx/Apache access, app JSON logs, or custom text) - What I want to measure (request count by status, p50/p95 latency, error rate, bytes) - Log rotation scheme (logrotate copytruncate vs rename+create, size, frequency) Deliver: 1. **Parser choice** — pick `data_format` (`grok`, `json_v2`, `logfmt`, `csv`) for my format and justify it. If grok, write the `grok_patterns`, any `custom_patterns`, and the named captures with type suffixes (`:int`, `:float`, `:ts- "layout"`). 2. **Tags vs fields** — map which captures become tags (method, status_class, host) and which stay fields (latency, bytes). Explicitly call out any capture that would blow up cardinality if tagged and how to bucket it instead. 3. **Multiline** — if stack traces span lines, configure `[inputs.tail.multiline]` (`pattern`, `match_after`/`before`, `invert_match`, `timeout`). 4. **Rotation & offsets** — set `watch_method` (inotify vs poll), `from_beginning`, and explain offset persistence so a Telegraf restart does not re-ingest or drop lines. Address copytruncate vs create explicitly. 5. **Filtering** — use `grok` failure handling / `namepass`/`tagpass` or a `filter` so unpar. seable lines don't spam errors, and drop health-check noise. 6. **Timestamps** — extract the log's own timestamp into the metric time (not ingest time) and handle timezone. Output: (a) a commented `inputs.tail` TOML block, (b) the grok/json_v2 config, (c) a cardinality note per tag, and (d) a `telegraf --test --config` command plus 3 sample lines to validate parsing before deploy. Bias toward: parsing the log's own timestamp, bounded tag sets, and rotation-safe offset handling.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Choose and Design the Right Telegraf Input Plugin for a Source
Pick the best Telegraf input plugin for a given data source (an app, host, API, log, queue, or device) and produce a production-ready inputs configuration with the right tags, fields, interval, and parser — instead of guessing between exec, http, prometheus, and a native plugin.
-
Telegraf Docker Input Metrics Collection Prompt
Configure inputs.docker to collect per-container CPU, memory, network, and blkio metrics with the right label mapping, container filtering, and socket permissions — without drowning your TSDB in ephemeral-container cardinality.
-
Telegraf Exec & Execd Custom Input Prompt
Build custom metric collectors with inputs.exec (run a script per interval) and inputs.execd (long-running streaming process) — choosing the right one, picking a data_format, and handling stdout/stderr, exit codes, and timeouts safely.
-
Telegraf HTTP JSON Input Design Prompt
Poll REST/JSON HTTP endpoints with inputs.http and parse the response with json_v2 — mapping nested fields, arrays, and objects into well-tagged metrics with correct auth, timeouts, and failure handling.
More Telegraf prompts & error guides
Browse every Telegraf prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.