🚨 Filebeat Error Guide: 'connection refused' — Restore the Elasticsearch Output

Overview

Filebeat logs this when its Elasticsearch output cannot open a TCP connection to the configured host. The publisher pipeline backs off and retries, so the message repeats on a growing interval while events queue in memory:

Failed to connect to backoff(elasticsearch(https://es01:9200)): dial tcp 10.0.3.14:9200: connect: connection refused

connection refused is an explicit TCP RST from the destination: something answered at that IP but nothing is listening on that port, or a firewall is actively rejecting rather than dropping the packet. Filebeat never gets as far as sending a bulk request — the socket handshake itself fails. Until the output becomes reachable, no events are indexed and Filebeat holds them in the internal queue (and the registry keeps the file offsets, so nothing is lost yet).

Symptoms

• Filebeat logs repeated Failed to connect to backoff(elasticsearch(...)): connection refused with an increasing backoff delay.
• filebeat test output fails at the connection... line with dial tcp ...: connect: connection refused.
• No new documents appear in the target data stream or index; document counts are flat in Kibana.
• The internal queue fills; on busy hosts you eventually see queue is full or harvesters pause.
• journalctl -u filebeat shows the pipeline connecting, failing, and reconnecting in a loop.

Common Root Causes

• Wrong port or host — output.elasticsearch.hosts points at 9200 when the cluster listens elsewhere, or at a hostname that resolves to the wrong node.
• Elasticsearch not running — the node is down, still starting, or crashed after an OOM; nothing is bound to the port.
• Scheme mismatch — the output uses http:// against a TLS-only node (or vice versa), so the listener rejects the connection.
• Firewall / security group — a host firewall or cloud security group rejects the port instead of allowing it.
• Bound to localhost only — Elasticsearch network.host is 127.0.0.1, so remote Filebeat hosts get refused.
• Load balancer with no healthy backends — the VIP answers but forwards to a pool with zero live nodes.

Diagnostic Workflow

Start with Filebeat’s own output test, which exercises exactly the configured hosts, scheme, and credentials:

filebeat test output
filebeat test config -c /etc/filebeat/filebeat.yml

Confirm what host and port Filebeat is actually using:

grep -A6 'output.elasticsearch' /etc/filebeat/filebeat.yml

Reach Elasticsearch directly from the Filebeat host to isolate network from application:

curl -v http://es01:9200            # watch for "Connection refused" vs TLS errors
curl -sk https://es01:9200 -u elastic:$ES_PASS | head

Check whether anything is listening on the ES node itself:

ss -ltnp | grep 9200
curl -s localhost:9200/_cluster/health?pretty

Verify DNS and raw reachability from the Filebeat host:

getent hosts es01
nc -vz es01 9200

If nc also reports refused but ss on the ES node shows it listening, the gap is a firewall between the two hosts.

Example Root Cause Analysis

A team migrated Elasticsearch behind a TLS-terminating proxy and switched the listener to 9200/https only, but the Filebeat config still read:

output.elasticsearch:
  hosts: ["http://es01:9200"]

filebeat test output reported connection refused, yet ss -ltnp on es01 clearly showed the port bound. The tell was curl -v http://es01:9200 failing immediately while curl -sk https://es01:9200 succeeded: the node now spoke only TLS, so plaintext connections were reset. Correcting the scheme and adding the CA fixed it:

output.elasticsearch:
  hosts: ["https://es01:9200"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca.crt"]

After systemctl restart filebeat, filebeat test output returned talk to server... OK and documents began flowing within seconds — the queued events drained because the registry had preserved every offset.

Prevention Best Practices

• Pin output.elasticsearch.hosts with the correct scheme and use a load balancer or multiple hosts so a single node restart does not stop ingestion.
• Add a startup check (filebeat test output) to your provisioning so a bad host/port is caught before deploy, not in production.
• Set Elasticsearch network.host to a routable address (or 0.0.0.0 behind a firewall) and confirm the security group allows the Filebeat subnet.
• Alert on Filebeat’s libbeat.output.events.failed and reconnect metrics so a refused output pages you before the queue backs up.
• Keep the CA and scheme in configuration management so TLS cutovers update Filebeat and Elasticsearch together.

Quick Command Reference

filebeat test output                                   # exercise the configured output
filebeat test config -c /etc/filebeat/filebeat.yml     # validate config syntax
curl -v http://es01:9200                               # plaintext reachability
curl -sk https://es01:9200 -u elastic:$ES_PASS         # TLS reachability
ss -ltnp | grep 9200                                   # is ES listening on the node?
nc -vz es01 9200                                        # raw TCP reach from Filebeat host
journalctl -u filebeat -f                              # watch reconnect loop

Conclusion

connection refused is a transport-layer failure, not a Filebeat bug: the port is wrong, the scheme is wrong, the node is down, or a firewall rejects the packet. Work from Filebeat outward — filebeat test output, then curl to the node, then ss on the node itself — and the layer that fails tells you exactly what to fix. Because Filebeat holds offsets in the registry, fixing the output drains the backlog with no data loss. See more fixes in the Filebeat guides.