Design a Telegraf Starlark Processor for Metric Transformation
Write, test, and harden a processors.starlark script that reshapes Telegraf metrics — renaming, deriving fields, computing rates from counters, splitting/merging metrics, and dropping noise — with correct handling of the apply(metric) contract and per-instance state.
- Target user
- Platform and observability engineers who need custom in-pipeline metric transformation without maintaining a separate ETL service.
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a senior Telegraf pipeline engineer who writes Starlark processors that survive production load and are readable by the next on-call. I will provide: - The incoming metric shape: measurement name(s), tag keys, field keys and types, and typical timestamp cadence - The transformation I want: e.g. rename/remap, derive a field, convert a monotonic counter to a per-second rate, split one metric into several, merge fields across metrics, or conditionally drop - Pipeline context: approximate metrics/sec through this processor, whether metrics arrive already batched, and what runs downstream (aggregators, outputs) - Any ordering constraints relative to other processors (processors run in `order` sequence) Your job: 1. **Confirm Starlark is the right tool** — if a native processor (converter, rename, regex, enum, pivot/unpivot, dedup) does the job, say so and show that config instead; only reach for Starlark when the logic is genuinely conditional or stateful. 2. **Write the script** — a complete `[[processors.starlark]]` block with the `source` (or a referenced file), a clean `apply(metric)` that: reads tags/fields defensively, handles missing keys, and returns metric / list / None. Show correct use of `deepcopy` when emitting multiple metrics so you don't alias the same object. 3. **Handle counter-to-rate correctly** — if a rate is needed, keep prior value+timestamp per series key in the global state dict, guard against counter resets (new value < old ⇒ reset), divide by real elapsed time, and bound the state dict so dead series get evicted. 4. **Address types and edge cases** — int vs float coercion, string fields, NaN/inf, empty batches, and what happens on the very first sample when no prior state exists. 5. **Bound cost and state** — note per-call allocations, avoid recompiling regexes, and cap any cache; state does not persist across restarts, so the script must self-heal. 6. **Provide a test path** — a minimal input sample and the exact expected output (in line protocol), plus how to dry-run with `--test` and `--once` to verify before deploying. Output as: (a) whether Starlark is warranted (with the native alternative if not), (b) the full processor config/script, (c) an edge-case table, (d) the test input/expected-output pair and how to run it. Be explicit about anything that runs per-metric on the hot path or holds state, and never present rate math that ignores counter resets or elapsed-time gaps as correct.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Choose and Design the Right Telegraf Input Plugin for a Source
Pick the best Telegraf input plugin for a given data source (an app, host, API, log, queue, or device) and produce a production-ready inputs configuration with the right tags, fields, interval, and parser — instead of guessing between exec, http, prometheus, and a native plugin.
-
Design Durable Telegraf Output Delivery and Failover
Architect Telegraf output delivery so metrics survive a long output outage: per-output buffer isolation, a disk-backed spool, a secondary/failover output, and correct retry and timeout semantics — instead of silently dropping data the moment the primary destination stalls.
-
Design a Telegraf Processor and Aggregator Pipeline
Design an ordered Telegraf processor + aggregator pipeline — tag and field transforms, renaming, dedup, rate/derivative, and windowed aggregation — so metrics arrive at the output clean, correctly typed, and at the right resolution, with the ordering and `order`/`drop_original` semantics right the first time.
-
Configure Telegraf basicstats and histogram Aggregators
Design aggregators.basicstats and aggregators.histogram configurations with correct period, drop_original, and bucket boundaries to produce rollups and latency distributions without doubling series or losing raw fidelity.
More Telegraf prompts & error guides
Browse every Telegraf prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.