🚨 Filebeat Error: 'circuit_breaking_exception: Data too large' — Cause, Fix, and Troubleshooting Guide

Overview

Elasticsearch protects itself with circuit breakers. When a bulk request would push heap usage past the parent breaker limit, ES rejects it and Filebeat surfaces the rejection on its bulk items:

Failed to perform any bulk index operations: 429 Too Many Requests: {"error":{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [8123456789/7.5gb], which is larger than the limit of [7654321098/7.1gb]","bytes_wanted":8123456789,"bytes_limit":7654321098,"durability":"TRANSIENT"}}

This is a cluster-side memory-protection response, not a Filebeat bug. The breaker trips because the node is under heap pressure — often from oversized bulk requests, large mappings, or too much concurrent work. Filebeat treats it as retryable backpressure and resends, so events are not lost, but ingestion stalls until the heap situation eases.

Symptoms

• circuit_breaking_exception with [parent] Data too large in the Filebeat bulk error.
• Bursts of 429/retry with libbeat.output.events.failed climbing.
• Elasticsearch node logs show [parent] breaker trips and high heap_used_percent.
• Correlated with large events, big bulk_max_size, or a memory-starved data node.

Common Root Causes

• Heap pressure on the node — heap_used_percent near the breaker limit (default 95% parent).
• Oversized bulk requests — a large bulk_max_size sending multi-megabyte payloads.
• Too many parallel workers — high worker count multiplying concurrent request memory.
• Undersized data nodes — heap too small for the mapping/shard count and event rate.
• Field explosion — enormous documents or too many fields inflating request memory.

How to diagnose

Check breaker state and heap on the cluster:

curl -sk 'https://es01:9200/_nodes/stats/breaker?pretty' -u elastic:$ES_PASS | \
  grep -A4 '"parent"'
curl -sk 'https://es01:9200/_cat/nodes?v&h=name,heap.percent,ram.percent,cpu' \
  -u elastic:$ES_PASS

Watch Filebeat retry cadence to correlate with the breaker trips:

journalctl -u filebeat -f | grep -i 'circuit_breaking\|Data too large\|retry'

Fixes

Send smaller, steadier bulk requests from Filebeat:

output.elasticsearch:
  bulk_max_size: 500        # was larger; smaller payloads pressure heap less
  worker: 1                 # fewer concurrent in-flight requests
  backoff.init: 1s
  backoff.max: 60s

The durable fix is on Elasticsearch — relieve heap pressure by adding data nodes, right-sizing heap (≤50% RAM, ≤~31GB), and reducing shard count. Do not raise the breaker limit as a first resort; it exists to prevent OOM. As a temporary measure while scaling:

# Confirm the trip is transient (heap spike) vs sustained overload
curl -sk 'https://es01:9200/_cat/thread_pool/write?v&h=node_name,active,queue,rejected' \
  -u elastic:$ES_PASS

Restart Filebeat after tuning and confirm failed/retry stop climbing.

What to watch out for

• Raising indices.breaker.total.limit masks the problem and risks OOM — fix heap instead.
• Events are retried, not dropped, so this is recoverable once the cluster has headroom.
• Big bulk_max_size × high worker multiplies request memory; tune both together.
• Persistent trips mean genuine capacity shortfall, not a transient spike.

Related

• Filebeat Error Guide: ‘429 Too Many Requests’
• Filebeat Error Guide: ‘413 Request Entity Too Large’
• Filebeat Error Guide: ‘temporary bulk send failure’