🚨 Kafka Error Guide: 'Error while accepting connection' SocketServer Processor Failure

Exact Error Message

These errors appear in the broker log (not the client) when Kafka’s SocketServer cannot accept or process incoming connections:

[2026-06-29 03:27:51,402] ERROR Error while accepting connection (kafka.network.Acceptor)
java.io.IOException: Too many open files
        at java.base/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at java.base/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:298)
        at kafka.network.Acceptor.accept(SocketServer.scala:705)
        at kafka.network.Acceptor.run(SocketServer.scala:647)
[2026-06-29 03:27:51,419] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.io.IOException: Too many open files

A bind-failure variant prevents the broker from starting at all:

[2026-06-29 03:25:10,883] ERROR [KafkaServer id=2] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Socket server failed to bind to 0.0.0.0:9092: Address already in use.
        at kafka.network.Acceptor.openServerSocket(SocketServer.scala:746)

What the Error Means

The SocketServer is the broker component that owns the listening sockets. An Acceptor thread accepts new TCP connections on each listener; Processor threads then read requests and write responses. These errors mean that machinery itself failed — not that one client misbehaved.

Error while accepting connection / Processor got uncaught exception with “Too many open files” means the broker process hit its file-descriptor (fd) ceiling. Every socket, log segment, and index file consumes an fd. When the limit is reached, the kernel refuses new accept() calls, so all new clients are turned away even though the broker is “up.” This degrades the whole broker, not a single connection.

Socket server failed to bind is different: it happens at startup, before any client connects. The broker could not claim the listener address/port — usually because something already holds it or the address is invalid.

Common Causes

• File-descriptor limit too low. The default nofile ulimit (often 1024) is far below Kafka’s needs (recommended 100000+). Under connection or partition growth, the broker exhausts fds.
• Connection leak / churn. Thousands of short-lived or leaked client connections inflate the open-fd count faster than they close.
• Port already in use. A stale Kafka process, a second broker instance, or another service holds 9092/9093, so the new broker cannot bind.
• Invalid or unreachable listener/advertised address. Binding to an IP not present on the host fails with “Cannot assign requested address.”
• Too many partitions/segments per broker raising the steady-state fd count toward the limit.

How to Reproduce the Error

Lower the fd limit and open many connections. With a broker started under a tight limit:

# broker launched under: ulimit -n 256
for i in $(seq 1 400); do kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic t --timeout-ms 60000 & done

The broker log will fill with Too many open files from the Acceptor. To reproduce the bind failure, start a second broker on a port the first already owns; it fails with Address already in use.

Diagnostic Commands

All read-only.

Check the broker process’s fd limit and current usage:

cat /proc/$(pgrep -f kafka.Kafka)/limits | grep -i 'open files'
ls /proc/$(pgrep -f kafka.Kafka)/fd | wc -l

Scan the broker log for the failure mode:

sudo journalctl -u kafka --no-pager --since "30 min ago" | grep -iE 'Too many open files|accepting connection|Processor got uncaught|failed to bind|Address already in use'

See who already holds the listener port (bind failures):

sudo ss -ltnp | grep -E ':9092|:9093'

Confirm the address Kafka is trying to bind actually exists on the host:

ip -br addr | grep -E '10\.0\.|9092' ; getent hosts $(hostname)

Count current connections per broker port to spot churn/leaks:

ss -tn state established '( sport = :9092 )' | wc -l

Step-by-Step Resolution

1. Identify which failure you have. “Too many open files” = fd exhaustion at runtime. “Address already in use” / “Cannot assign requested address” = bind failure at startup. They have different fixes.

2. For fd exhaustion, raise the limit. Check /proc/<pid>/limits. If nofile is low, raise it for the Kafka user/service. For a systemd unit, set LimitNOFILE=100000 in the unit and reload; for init/limits.conf, set kafka soft/hard nofile 100000. Restart the broker so it inherits the new limit, then re-check /proc/<pid>/limits.

3. Investigate connection leaks. If fds climb steadily under stable load, count established connections (ss above). A client that opens connections without closing, or an absent connections.max.idle.ms, can leak. Fix the offending client and ensure idle connections are reaped.

4. For bind failures, free the port. sudo ss -ltnp | grep :9092 reveals the holder. If it is a stale Kafka process, stop it cleanly before restarting. If two brokers share a host, give each a distinct port.

5. For “Cannot assign requested address,” fix the listener. Ensure listeners/advertised.listeners reference an IP/interface that exists on the host (or 0.0.0.0 for all). Verify with ip -br addr.

6. Reduce steady-state fd pressure if partition counts are very high: consolidate topics or add brokers so per-broker segment/fd counts drop.

Prevention and Best Practices

• Set LimitNOFILE to 100000 or higher for the Kafka service and verify it via /proc/<pid>/limits after every deploy — a silent default of 1024 is the most common cause.
• Monitor open-fd count and established-connection count per broker; alert well before the limit.
• Enforce a sane connections.max.idle.ms so abandoned client connections are reaped and do not leak fds.
• Make listeners/advertised.listeners explicit and validated against the host’s actual interfaces to avoid bind failures on startup.
• Cap partitions per broker within recommended limits so steady-state fd usage stays well under the ceiling.
• Ensure clean shutdowns so stale processes never hold the listener port across restarts.

Related Errors

• Connection to node -1 could not be established — the client-side symptom when the broker’s SocketServer cannot accept connections.
• Connection reset by peer / Broken pipe — clients see these when an overloaded broker drops their connections.
• Address already in use — the specific bind-failure flavor covered above.
• Node N disconnected — clients drop when the broker stops accepting on its listener.

Frequently Asked Questions

Is this a client error or a broker error? A broker error. SocketServer runs on the broker. Clients only see secondary symptoms (refused, reset, node -1).

What fd limit does Kafka need? Far more than the default 1024. 100000 (LimitNOFILE=100000) is a common recommended floor; high-partition brokers may need more.

Why does the broker stay “up” while rejecting everyone? The process is alive but cannot accept() new sockets once fds are exhausted. Existing connections may continue while new ones fail until fds free up or the limit is raised.

Why can’t the broker bind on startup? Another process (often a stale Kafka instance) holds the port, or the configured listener IP does not exist on the host. ss -ltnp and ip -br addr identify which.

Does adding partitions affect this? Yes. More partitions mean more open log segment/index files, raising steady-state fd usage toward the limit.