🚨 Logstash Error Guide: 'circuit_breaking_exception ... Data too large' — Relieve the Elasticsearch Memory Breaker

Overview

Elasticsearch protects its JVM heap with circuit breakers. When a bulk request from the Logstash elasticsearch output would push heap usage past the parent breaker limit, ES rejects it and Logstash surfaces the response:

[ERROR][logstash.outputs.elasticsearch] Encountered a retryable error (will retry with
exponential backoff) {:code=>429, :url=>"http://es:9200/_bulk", :body=>
"{\"error\":{\"type\":\"circuit_breaking_exception\",
\"reason\":\"[parent] Data too large, data for [<http_request>] would be
[8321499136/7.7gb], which is larger than the limit of [8160437862/7.5gb]\",
\"bytes_wanted\":8321499136,\"bytes_limit\":8160437862,
\"durability\":\"TRANSIENT\"}}"}

Data too large means the request, plus already-committed heap, exceeds the breaker’s limit. It is an Elasticsearch-side heap-pressure problem that the Logstash output merely reports — the fix is to reduce memory demand on ES, not just retry.

Symptoms

• circuit_breaking_exception ... Data too large with a bytes_wanted/bytes_limit pair in the Logstash log.
• The breaker named is usually [parent], sometimes [request] or [fielddata].
• Bulk requests fail intermittently under load and succeed when volume drops.
• ES data-node heap sits high; GC is frequent; _nodes/stats/breakers shows tripped > 0.
• Ingest lag grows as Logstash retries the rejected bulks with backoff.

Common Root Causes

• Bulk requests too large — a huge pipeline.batch.size produces multi-hundred-MB _bulk payloads that spike heap on arrival.
• ES heap under-provisioned or already pressured — search, aggregations, or fielddata leave little headroom for ingest.
• Fielddata on text fields — aggregations/sorts on analyzed fields load fielddata into heap and trip the breaker.
• Too many concurrent bulks — many Logstash output workers × large batches arrive simultaneously.
• Oversized documents — very large events (huge messages, big arrays) inflate per-request heap.
• Mapping explosion — thousands of dynamically created fields bloat cluster state and heap.

Diagnostic Workflow

Look at the Elasticsearch breakers to see which one trips and how close to the limit heap runs:

curl -s 'http://es:9200/_nodes/stats/breakers?pretty' | \
  grep -A6 -E 'parent|fielddata|request'
curl -s 'http://es:9200/_cat/nodes?v&h=name,heap.percent,heap.current,heap.max'

Estimate the bulk size Logstash is sending — batch size × average event size:

curl -s localhost:9600/_node/stats/pipelines?pretty | \
  grep -E '"in"|"out"|duration_in_millis'
grep -E 'pipeline.batch.size' /etc/logstash/pipelines.yml

Shrink the bulk payload in pipelines.yml:

- pipeline.id: main
  path.config: "/etc/logstash/conf.d/main.conf"
  pipeline.workers: 4
  pipeline.batch.size: 250      # smaller _bulk payload => lower ES heap spike

Check whether fielddata is the breaker being tripped, and on which fields:

curl -s 'http://es:9200/_cat/fielddata?v&s=size:desc' | head
curl -s 'http://es:9200/_nodes/stats/indices/fielddata?fields=*&pretty' | head -40

If ES heap is genuinely undersized, inspect and plan to raise it (ES jvm.options, kept ≤ 50% RAM and under ~30 GB for compressed oops):

curl -s 'http://es:9200/_cat/nodes?v&h=name,heap.max,ram.max'

Confirm the pipeline recovers once bulks shrink or heap frees:

tail -f /var/log/logstash/logstash-plain.log | grep -Ei 'circuit_breaking|retry'

Example Root Cause Analysis

An ingest cluster with 8 GB heap per data node ran fine overnight but tripped [parent] Data too large every morning when a scheduled Kibana dashboard fired heavy aggregations. Simultaneously, Logstash was configured with pipeline.batch.size: 4000 across 6 output workers, producing ~200 MB _bulk payloads.

_nodes/stats/breakers showed the parent breaker at 95% during the aggregation window, and each large bulk arriving on top of that tipped it over. The immediate relief was to drop pipeline.batch.size to 500, cutting per-request heap spikes. The durable fix addressed the aggregation side: the offending dashboard sorted on an analyzed text field, loading fielddata; switching it to the .keyword sub-field eliminated the fielddata heap load. With both changes, breaker usage stayed under 70% and the Data too large rejections stopped.

Prevention Best Practices

• Keep pipeline.batch.size moderate so _bulk payloads stay well under the breaker limit; scale ingest horizontally, not with giant bulks.
• Size Elasticsearch heap to ≤ 50% of RAM and under ~30 GB (compressed oops), and leave headroom for ingest on top of search load.
• Never aggregate or sort on analyzed text fields — use .keyword sub-fields to avoid fielddata heap pressure.
• Control mapping explosion with strict templates and index.mapping.total_fields.limit; thousands of dynamic fields bloat heap.
• Monitor _nodes/stats/breakers tripped and node heap.percent; alert before the parent breaker approaches its limit.
• Separate ingest and heavy-search workloads (dedicated nodes or time windows) so aggregations do not steal heap from indexing.

Quick Command Reference

# Which breaker trips, and heap headroom
curl -s 'http://es:9200/_nodes/stats/breakers?pretty' | grep -A6 parent
curl -s 'http://es:9200/_cat/nodes?v&h=name,heap.percent,heap.max'

# What bulk size is Logstash sending?
grep pipeline.batch.size /etc/logstash/pipelines.yml
curl -s localhost:9600/_node/stats/pipelines?pretty | grep -E '"in"|"out"'

# Fielddata pressure
curl -s 'http://es:9200/_cat/fielddata?v&s=size:desc' | head

# Watch for recovery
tail -f /var/log/logstash/logstash-plain.log | grep -i circuit_breaking

Conclusion

circuit_breaking_exception: Data too large is Elasticsearch refusing a bulk because accepting it would exceed a JVM heap breaker — a memory-pressure problem on ES that the Logstash output only reports. Relieve it on both sides: shrink pipeline.batch.size and output concurrency so bulks are small, and cut ES heap demand by avoiding fielddata on text fields, controlling mapping explosion, and sizing heap properly. Monitor the parent breaker’s tripped counter and node heap so you scale before ingest starts failing, rather than after.