🚨 OpenTelemetry Error Guide: 'sending queue is full' — Fix OTLP Exporter Queue Overflow

Overview

The Collector’s exporter helper buffers outgoing telemetry in an in-memory sending queue. When the queue fills faster than the exporter can drain it, new batches are rejected and the Collector logs:

Exporting failed. Rejecting data.
error: sending queue is full
rejected_items: 8192

You will also see the paired dropped-data message as the batches are discarded:

Dropping data because sending queue is full. Try increasing queue_size.
kind: exporter  data_type: traces  name: otlp

Once the queue is full, data is dropped at the exporter — this is backpressure, not a transient retry.

Symptoms

• Telemetry gaps during traffic spikes while steady-state is fine.
• Logs repeat sending queue is full and Dropping data because sending queue is full.
• otelcol_exporter_queue_size sits at or near otelcol_exporter_queue_capacity.
• otelcol_exporter_enqueue_failed_spans (or metrics/logs) is climbing.
• Correlates with backend slowness, throttling, or exporter retries backing up.

Common Root Causes

• Backend slower than ingest — the destination throttles or is slow, so the queue can’t drain.
• Too few consumers — num_consumers is too low to keep up with enqueue rate.
• Undersized queue — queue_size is too small to absorb bursts.
• Retry storms — a failing backend triggers retries that keep items in the queue longer, compounding pressure.
• No persistence — an in-memory queue can’t offload, so a brief backend outage overflows it.
• Single overloaded exporter — all volume funneled through one exporter with no horizontal scaling.

Diagnostic Workflow

Inspect the exporter’s queue and retry settings; these live under the exporter, not the batch processor:

exporters:
  otlp:
    endpoint: otel-backend:4317
    sending_queue:
      enabled: true
      num_consumers: 10      # parallel senders draining the queue
      queue_size: 5000       # batches buffered before rejecting
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s

Add a persistent queue backed by file storage so a backend blip doesn’t overflow memory and data survives restarts:

extensions:
  file_storage:
    directory: /var/lib/otelcol/queue

exporters:
  otlp:
    sending_queue:
      enabled: true
      storage: file_storage
      queue_size: 10000

service:
  extensions: [file_storage]

Confirm queue saturation and the backend as the bottleneck via the Collector’s own metrics:

curl -s http://localhost:8888/metrics | grep -E 'exporter_queue_size|exporter_queue_capacity|enqueue_failed'
journalctl -u otelcol -f | grep -i 'sending queue is full\|Dropping data'

Example Root Cause Analysis

A gateway Collector started dropping spans every weekday at 09:00 with sending queue is full. Metrics showed otelcol_exporter_queue_size pinned at the queue_capacity of 1000 during the morning ramp, while otelcol_exporter_send_failed_spans was near zero — the backend was accepting data, just not fast enough for the default queue during the burst.

The team raised queue_size to 10000 and num_consumers from 1 to 10 so more batches sent in parallel, then added a file_storage-backed persistent queue so a backend hiccup would spill to disk instead of dropping. After the change, the morning queue peaked at ~3000 and drained within seconds, with zero dropped spans. They also added an alert on queue_size / queue_capacity > 0.8.

Prevention Best Practices

• Size queue_size to absorb your worst realistic burst, and raise num_consumers so the queue drains in parallel.
• Enable a persistent file_storage queue on gateways so backend outages and restarts don’t drop in-flight data.
• Fix the real bottleneck: if the backend throttles, scale it or add more Collector replicas rather than only enlarging the queue.
• Alert on otelcol_exporter_queue_size / queue_capacity and on enqueue_failed_* so overflow is caught before data loss.
• Load-test at peak so queue sizing is validated in staging, not discovered in production.

Quick Command Reference

# Watch queue saturation
curl -s http://localhost:8888/metrics | grep -E 'exporter_queue_size|exporter_queue_capacity'

# Watch for drops live
journalctl -u otelcol -f | grep -i 'sending queue is full'

# Confirm enqueue failures by signal
curl -s http://localhost:8888/metrics | grep enqueue_failed

# Validate config after editing queue settings
otelcol validate --config /etc/otelcol/config.yaml

Conclusion

sending queue is full is backpressure: the exporter can’t drain telemetry as fast as it arrives, so the Collector drops data. Enlarge queue_size, add num_consumers, and back the queue with file_storage for durability — but also fix the underlying cause, whether that’s a slow backend or an overloaded single exporter. Alerting on queue saturation turns silent drops into an early warning.