Skip to content
DevOps AI ToolKit
Newsletter
All prompts
AI for OpenTelemetry Difficulty: Advanced ClaudeChatGPTCursor

OpenTelemetry Collector Batching & Memory Limiter Tuning Prompt

Tune the OpenTelemetry Collector's batch and memory_limiter processors plus exporter queues to maximize throughput and avoid OOMs, backpressure stalls, and dropped telemetry under bursty load.

Target user
Engineers tuning Collector throughput and stability
Difficulty
Advanced
Tools
Claude, ChatGPT, Cursor

The prompt

You are a senior observability engineer who tunes OpenTelemetry Collector throughput and memory stability under production load.

I will provide:
- Collector version, deployment mode, and per-instance CPU/memory requests and limits
- Ingest volume and burstiness per signal (average and peak spans/datapoints/logs per second)
- Current batch, memory_limiter, and exporter sending_queue/retry config
- Symptoms (OOMKilled, refused_spans, exporter queue full, latency spikes, drops)

Your job:

1. **Diagnose** — map each symptom to a cause using the Collector's own metrics (otelcol_processor_refused_*, otelcol_exporter_queue_size, otelcol_processor_batch_batch_send_size, memory_limiter refusals, dropped spans).
2. **memory_limiter** — set limit_mib and spike_limit_mib relative to the container limit and Go runtime overhead, and set check_interval; explain the safety margin and GOMEMLIMIT interaction.
3. **batch** — tune send_batch_size, send_batch_max_size, and timeout to balance throughput, backend batch-size limits, and latency; give values matched to my volume.
4. **Exporter queue** — size sending_queue (num_consumers, queue_size) and retry_on_failure so bursts are absorbed without unbounded memory, and decide whether file_storage persistence is warranted.
5. **Ordering** — confirm the pipeline order (memory_limiter first, batch last) and correct it if wrong.
6. **Capacity** — recommend CPU/memory requests/limits and horizontal scaling (more gateway replicas) vs vertical, with the trigger metric for scaling.
7. **Change plan** — give a one-variable-at-a-time tuning procedure with the metric to watch after each change.

Output as: (a) a symptom-to-cause table, (b) the tuned processor and queue YAML with the reasoning behind each number, (c) resource and scaling recommendations, (d) the metrics to alert on, (e) a safe iterative tuning procedure.

Explicitly flag any current setting that risks OOM, silent drops, or backpressure stalls, and the order to fix them.

Run this prompt with AI

Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.

Related prompts

More OpenTelemetry prompts & error guides

Browse every OpenTelemetry prompt and troubleshooting guide in one place.

Free download · 368-page PDF

Reading prompts? Get all 500 in one free PDF

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.