🚨 Filebeat Error: 'Failed to publish events caused by: EOF' to Logstash — Cause, Fix, and Troubleshooting Guide

Overview

When Filebeat ships to a logstash output over the Beats/Lumberjack protocol and Logstash closes the socket while Filebeat is mid-transfer, the read returns EOF and the batch fails:

Failed to publish events caused by: EOF
Failed to publish events caused by: client is not connected

EOF means the far end closed the connection cleanly (as opposed to connection reset by peer, which is an abrupt RST). With the Beats input this typically happens when Logstash’s pipeline back-pressures and the input drops slow clients, when client_inactivity_timeout fires, when TLS is misconfigured, or when Logstash restarts/reloads. Filebeat re-queues the batch and reconnects with backoff, so events are not lost, but throughput sawtooths and the log fills with EOFs.

Symptoms

• Recurring Failed to publish events caused by: EOF (and client is not connected) to a logstash output.
• Throughput stalls then recovers as Filebeat reconnects.
• Logstash logs show pipeline back-pressure, Beats input closing connections, or reloads.
• Correlates with Logstash CPU/pipeline saturation or periodic config reloads.

Common Root Causes

• Logstash pipeline back-pressure — a slow filter/output causes the Beats input to close clients.
• client_inactivity_timeout on the Beats input closing idle Filebeat connections.
• Logstash restarts / auto-reload dropping all connections.
• TLS mismatch — Filebeat expects TLS but the input is plaintext (or vice versa).
• Undersized Logstash — too few pipeline workers for the event rate.

How to diagnose

Watch both sides at the same time:

journalctl -u filebeat -f | grep -i 'EOF\|not connected\|logstash'
# On the Logstash host:
journalctl -u logstash -f | grep -i 'beats\|pipeline\|reset\|backpressure'

Confirm the Beats port is reachable and check TLS expectations:

nc -vz logstash01 5044
grep -A6 'output.logstash' /etc/filebeat/filebeat.yml

Fixes

Relieve Logstash back-pressure — add pipeline workers and fix the slow downstream output so the Beats input stops dropping clients:

# logstash.yml
pipeline.workers: 8
pipeline.batch.size: 250

Raise the Beats input inactivity timeout so healthy-but-quiet connections are not closed:

input {
  beats {
    port => 5044
    client_inactivity_timeout => 120   # default 60s
  }
}

On the Filebeat side, spread load across Logstash nodes and keep sane backoff:

output.logstash:
  hosts: ["logstash01:5044", "logstash02:5044"]
  loadbalance: true
  ttl: 30s
  backoff.init: 1s
  backoff.max: 60s

Make the TLS setting match on both ends (both ssl.enabled or both plaintext), then restart and confirm EOFs subside.

What to watch out for

• EOF is a clean close (often back-pressure/timeout); connection reset by peer is an abrupt RST — different root causes.
• The real fix is usually on Logstash: relieve pipeline back-pressure, do not just retry harder.
• A TLS/plaintext mismatch produces immediate EOFs on every connect.
• Frequent Logstash reloads drop connections; batch config changes instead.

Related

• Filebeat Error Guide: ‘connection reset by peer’ to Logstash
• Filebeat Error Guide: ‘connection refused’
• Filebeat Error Guide: ‘temporary bulk send failure’