🚨 Filebeat Error Guide: '413 Request Entity Too Large' — Shrink the Bulk Request

Overview

Elasticsearch (or a proxy in front of it) rejects a _bulk request whose body exceeds the allowed size, returning HTTP 413. Filebeat surfaces the rejection while trying to publish a batch:

Failed to perform any bulk index operations: 413 Request Entity Too Large: {"error":{"type":"illegal_argument_exception","reason":"entity content is too long [110692352] for the configured buffer limit [104857600]"}}

Elasticsearch caps HTTP request bodies at http.max_content_length (default 100 MB). A load balancer, reverse proxy (Nginx client_max_body_size), or API gateway may impose a smaller cap and return 413 first. When Filebeat’s assembled bulk batch — bulk_max_size events, some possibly huge — exceeds that limit, the whole request is rejected. Filebeat retries, but the same oversized batch fails identically, so ingestion stalls until the batch is made smaller or the limit is raised.

Symptoms

• Bulk 413 Request Entity Too Large in the Filebeat log, often citing a byte length vs. a buffer limit.
• The same batch fails repeatedly — retrying does not help because the size is unchanged.
• Ingestion for a particular source stalls, especially one that emits very large events.
• A proxy access log shows 413 responses before the request reaches Elasticsearch.
• Errors correlate with a spike in event size (e.g. stack traces, base64 blobs, multiline dumps).

Common Root Causes

• bulk_max_size too high — many events per request push the body over the limit.
• Oversized individual events — a single huge document (large stack trace, embedded payload) alone exceeds the cap.
• Proxy body limit — an Nginx/HAProxy/gateway client_max_body_size lower than Elasticsearch’s own limit.
• Lowered http.max_content_length — a cluster configured below the default for memory safety.
• Unbounded multiline — a misconfigured multiline pattern concatenates thousands of lines into one event.
• No event size cap — max_bytes on the input not set, so pathological lines pass through whole.

Diagnostic Workflow

Read the Filebeat log to get the actual body size versus the configured limit:

journalctl -u filebeat --since '20 min ago' | grep -i '413\|too long\|content is too long'

Check Elasticsearch’s configured content-length limit:

curl -sk 'https://es01:9200/_cluster/settings?include_defaults=true&pretty' \
  -u elastic:$ES_PASS | grep -A1 max_content_length

Reproduce with a deliberately large body to confirm where 413 originates (ES vs. proxy):

head -c 130000000 /dev/zero | tr '\0' 'x' > /tmp/big.txt
curl -sk -o /dev/null -w '%{http_code}\n' -X POST -u elastic:$ES_PASS \
  -H 'Content-Type: application/x-ndjson' 'https://es01:9200/_bulk' \
  --data-binary @/tmp/big.txt

Find the largest events being shipped and check the current bulk setting:

grep -E 'bulk_max_size|max_bytes|multiline' /etc/filebeat/filebeat.yml
curl -sk 'https://es01:9200/filebeat-*/_search?pretty' -u elastic:$ES_PASS \
  -H 'Content-Type: application/json' -d '{"size":1,"sort":[{"_size":"desc"}]}' 2>/dev/null | head

Example Root Cause Analysis

A service that logged full HTTP payloads occasionally emitted multi-megabyte lines. Combined with bulk_max_size: 3200, Filebeat assembled bulk bodies over 100 MB and the cluster returned 413 ... content is too long [110692352] for the configured buffer limit [104857600]. Because Filebeat retried the identical batch, that source’s ingestion froze.

Two fixes applied together. Filebeat batched fewer events and capped individual line size so a single monster event could not blow the limit:

output.elasticsearch:
  bulk_max_size: 500
filebeat.inputs:
  - type: filestream
    paths: ["/var/log/app/*.log"]
    message_max_bytes: 1048576     # drop/truncate lines over 1 MB

They also verified the Nginx proxy in front of Elasticsearch used client_max_body_size 100m; to match the cluster’s http.max_content_length, so the proxy would not 413 first. After the change, bulk bodies stayed well under the limit and the stalled source resumed.

Prevention Best Practices

• Keep bulk_max_size moderate so assembled bodies stay comfortably under http.max_content_length.
• Cap event size at the input (message_max_bytes) so a single pathological line cannot exceed the limit.
• Align every hop’s body limit — Elasticsearch http.max_content_length and any proxy client_max_body_size — so no layer 413s unexpectedly.
• Fix multiline patterns so they cannot concatenate unbounded numbers of lines into one event.
• Monitor event and bulk-body sizes; alert on outliers before they trip the limit.

Quick Command Reference

journalctl -u filebeat --since '20 min ago' | grep -i 'too long\|413'
curl -sk 'https://es01:9200/_cluster/settings?include_defaults=true&pretty' -u elastic:$ES_PASS | grep max_content_length
grep -E 'bulk_max_size|message_max_bytes|multiline' /etc/filebeat/filebeat.yml
curl -sk -o /dev/null -w '%{http_code}\n' -X POST 'https://es01:9200/_bulk' -H 'Content-Type: application/x-ndjson' -u elastic:$ES_PASS --data-binary @/tmp/big.txt

Conclusion

413 Request Entity Too Large means Filebeat’s bulk body exceeded Elasticsearch’s http.max_content_length or a stricter proxy limit — and since retrying the same batch cannot help, ingestion stalls until the body shrinks. Reduce bulk_max_size, cap individual events with message_max_bytes, fix runaway multiline, and align every proxy’s body limit with the cluster. Get the request under the cap and the source resumes. More output tuning is in the Filebeat guides.