🚨 Filebeat Error: 'kafka: client has run out of available brokers to talk to' — Cause, Fix, and Troubleshooting Guide

What this error means

Filebeat’s kafka output bootstraps by connecting to the brokers you list, then discovers the full broker set from metadata. When it cannot reach any broker — or the addresses it learns are unreachable — the underlying client gives up:

Failed to connect to broker kafka01:9092: dial tcp: lookup kafka01 on 10.0.0.2:53: no such host
kafka: client has run out of available brokers to talk to: EOF

The tell is that Filebeat exhausted every broker it knows and none accepted a connection. The usual culprits are wrong bootstrap addresses, brokers advertising internal hostnames the Filebeat host cannot resolve, or a TLS/SASL handshake that fails so every broker is marked unavailable. No events publish until at least one broker is reachable and the handshake succeeds.

How the agent reports it

• kafka: client has run out of available brokers to talk to (often with EOF or dial tcp before it).
• Failed to connect to broker <host>:<port> for each configured broker.
• Works from the broker host but not from the Filebeat host (advertised-listener mismatch).
• Started after enabling TLS/SASL or changing advertised.listeners on Kafka.

Checking the agent configuration

Confirm basic reachability from the Filebeat host:

nc -vz kafka01 9092
getent hosts kafka01

Run Filebeat with output debug to see the metadata/broker negotiation:

filebeat -e -d "kafka" -c /etc/filebeat/filebeat.yml

Check what the brokers advertise — a mismatch here is the classic cause:

kafka-broker-api-versions.sh --bootstrap-server kafka01:9092 2>&1 | head
# or inspect server.properties: advertised.listeners=

Agent configuration causes

• Wrong bootstrap hosts — unreachable address or port.
• Advertised listeners returning internal hostnames Filebeat cannot resolve.
• TLS handshake failure — untrusted CA, hostname mismatch, or plaintext-vs-TLS mismatch.
• SASL auth failure — wrong mechanism/credentials, marking brokers unavailable.
• Firewall/security group blocking 9092/9093 from the Filebeat host.
• Kafka down or the topic’s leader broker offline.

Step-by-step resolution

List reachable brokers and match the advertised-listener names your host can resolve:

output.kafka:
  hosts: ["kafka01:9092", "kafka02:9092", "kafka03:9092"]
  topic: "filebeat-logs"
  partition.round_robin:
    reachable_only: true
  required_acks: 1

If the brokers advertise internal names, add host entries or fix advertised.listeners on Kafka so the returned addresses resolve from the Filebeat host. For TLS:

output.kafka:
  hosts: ["kafka01:9093"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/kafka-ca.pem"]
  ssl.verification_mode: "full"

For SASL, set the mechanism and credentials the cluster expects:

output.kafka:
  username: "filebeat"
  password: "${KAFKA_PASS}"
  sasl.mechanism: "SCRAM-SHA-512"

Test the output after each change:

filebeat test output

Safer agent defaults

• The bootstrap hosts only start discovery; the advertised listeners must also be reachable.
• A plaintext client against a TLS listener (or vice versa) fails every broker with cryptic EOFs.
• reachable_only: true avoids stalling on a dead partition leader.
• SASL mechanism must match the broker exactly (PLAIN vs SCRAM-SHA-256/512).

Related agent errors

• Filebeat Error Guide: ‘connection refused’
• Filebeat Error Guide: ‘backoff(elasticsearch(…)): lookup: no such host’
• Filebeat Error: ‘Connection marked as failed because the onConnect callback failed’