🚨 Kafka Error Guide: 'NoNodeException for /brokers/ids/1' Missing ZooKeeper Znode

Exact Error Message

org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /brokers/ids/1
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:114)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
        at kafka.zookeeper.AsyncResponse.maybeThrow(ZooKeeperClient.scala:583)
        at kafka.zk.KafkaZkClient.getDataAndStat(KafkaZkClient.scala:1819)
        at kafka.zk.KafkaZkClient.getBroker(KafkaZkClient.scala:611)
        at kafka.admin.BrokerCommand.describe(BrokerCommand.scala:142)

You may see the same exception class for other paths, for example when an admin tool reads topic config or consumer offsets:

org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /config/topics/orders
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/payments-group/offsets/orders/0

What the Error Means

NoNodeException is ZooKeeper telling you that the znode (the ZooKeeper data path) you asked to read does not exist at the location you looked. Kafka stores its cluster metadata as a tree of znodes: /brokers/ids/<id> for each live broker, /config/topics/<topic> for per-topic config, /controller for the elected controller, and so on. When a broker or admin tool requests one of those paths and ZooKeeper has nothing there, you get NoNode.

The key insight is that this is a lookup against a specific path on a specific ensemble. The error almost never means your data is lost. It usually means you are looking in the wrong place: the wrong chroot, the wrong ZooKeeper ensemble, or a path that simply has not been created yet.

This guide applies only to legacy, ZooKeeper-based Kafka clusters (the classic deployment model). KRaft-mode clusters store metadata in an internal Kafka log managed by a quorum of controllers and do not use ZooKeeper at all, so NoNode for /brokers/ids/... cannot occur there. If you are on KRaft, the equivalent metadata lives in __cluster_metadata and you would diagnose it with kafka-metadata-quorum.sh instead.

Common Causes

• Wrong or missing chroot in zookeeper.connect. The single most common cause. A broker configured with zookeeper.connect=zk:2181/kafka writes all znodes under the /kafka chroot prefix, so its registration lives at /kafka/brokers/ids/1. A tool pointed at zk:2181 (no chroot) looks at the root /brokers/ids/1, finds nothing, and throws NoNode.
• Pointing tools at the wrong ZooKeeper ensemble. In an environment with staging and production ensembles, aiming an admin command at the wrong host:port set reads an ensemble that never held that broker’s metadata.
• A fresh cluster where znodes do not exist yet. On first startup, before any broker has registered or any topic has been created, the corresponding paths are simply absent.
• An accidentally deleted znode. Someone ran a destructive delete in zookeeper-shell.sh, or a cleanup script removed the path.
• Reading a path that genuinely does not exist. Querying /config/topics/orders for a topic that was never created, or a consumer group that has no committed offsets, returns NoNode by design.
• Mismatched chroot between brokers and admin tools. Brokers agree on /kafka, but an operator’s CLI defaults to the root, so every read misses.

How to Reproduce the Error

Set up a broker with a chroot and then read without it. With a broker started using zookeeper.connect=localhost:2181/kafka, the broker registers at /kafka/brokers/ids/1. Now point a tool at the bare root:

# Broker uses the /kafka chroot, but this tool omits it
zookeeper-shell.sh localhost:2181 ls /brokers/ids
# returns [] (empty) -> any read of /brokers/ids/1 yields NoNode

Equally, on a brand-new ensemble with no Kafka data at all, asking for any Kafka path reproduces it immediately because the tree has not been populated.

Diagnostic Commands

All commands below are read-only. They inspect the ZooKeeper tree and Kafka without changing anything.

# List the top-level znodes on the ensemble you are actually talking to
zookeeper-shell.sh zk1:2181 ls /

# If brokers use a chroot (e.g. /kafka), list under it
zookeeper-shell.sh zk1:2181 ls /kafka

# Where Kafka registrations should live - with and without chroot
zookeeper-shell.sh zk1:2181 ls /brokers/ids
zookeeper-shell.sh zk1:2181 ls /kafka/brokers/ids

# Inspect the controller and a specific broker registration (read-only get)
zookeeper-shell.sh zk1:2181 get /kafka/controller
zookeeper-shell.sh zk1:2181 get /kafka/brokers/ids/1

# Confirm the broker is actually up and answering on the Kafka side
kafka-broker-api-versions.sh --bootstrap-server kafka1:9092 | head -5

# Check what the broker thinks its zookeeper.connect is
grep -E '^zookeeper.connect' /etc/kafka/server.properties

# Broker and ZooKeeper logs around the failure
journalctl -u kafka --no-pager | grep -iE 'NoNode|zookeeper.connect|chroot' | tail -20
journalctl -u zookeeper --no-pager | tail -20

# Read-only four-letter words to confirm the ensemble is healthy
echo ruok | nc zk1 2181
echo stat | nc zk1 2181

If ls / shows your Kafka subtree under /kafka but ls /brokers/ids at the root is empty, you have just confirmed a chroot mismatch.

Step-by-Step Resolution

1. Identify the real chroot. Run grep -E '^zookeeper.connect' /etc/kafka/server.properties on a broker. If the value ends in a path such as /kafka, that path is the chroot and every Kafka znode lives beneath it.

2. Make every tool use the same ensemble and chroot. When invoking admin tools that talk to ZooKeeper, include the identical connection string the brokers use, chroot and all. Point CLI tools at the bootstrap servers (--bootstrap-server) where possible, since modern Kafka admin operations go through the brokers rather than directly into ZooKeeper.

3. Correct zookeeper.connect so brokers and tools agree. If brokers were split across different chroots by mistake, settle on one value (for example zk1:2181,zk2:2181,zk3:2181/kafka) and apply it consistently. After changing a broker’s zookeeper.connect, restart that broker so it re-registers under the agreed path.

4. Confirm you are on the right ensemble. If staging and production share host naming, double-check the resolved hosts and ports. Reading the wrong ensemble produces NoNode even when everything is healthy.

5. For a fresh cluster, let brokers create their own znodes. On first boot, start the brokers and let each one register itself. The /brokers/ids/<id> path appears automatically once a healthy broker connects. Do not pre-create these paths.

6. Do NOT manually create the znodes to “fix” the error. Hand-writing /brokers/ids/1 with create produces a permanent (non-ephemeral) node that does not reflect a live session, which leads to controller confusion and NodeExists errors later. The correct fix is always to point at the right path and let Kafka own its own registration.

After correcting the chroot or ensemble, re-run zookeeper-shell.sh ... ls /brokers/ids (or under the chroot) and confirm the broker IDs appear.

Prevention and Best Practices

• Standardize one zookeeper.connect string, chroot included, across every broker and every operator’s tooling, and store it in config management rather than typing it by hand.
• Prefer --bootstrap-server flags over direct ZooKeeper access for admin tasks so tools never need to know the chroot.
• Use a distinct chroot per cluster (for example /kafka-prod and /kafka-stage) on a shared ensemble so a wrong target fails obviously instead of silently reading the wrong data.
• Restrict and audit access to zookeeper-shell.sh so accidental delete operations cannot remove live znodes.
• Document, for each cluster, the ensemble hosts and the chroot in your runbook next to the bootstrap servers.
• Plan a migration to KRaft mode; it removes ZooKeeper entirely and with it this whole class of chroot and ensemble mismatch errors. Browse more Kafka troubleshooting guides for migration-adjacent issues.

Related Errors

• KeeperException$NodeExistsException: NodeExists for /brokers/ids/1 — the inverse problem: the registration path already exists, usually a duplicate broker.id or a stale ephemeral node.
• KeeperException$ConnectionLossException — the client lost its session to ZooKeeper; an ensemble or network problem rather than a missing path.
• org.apache.kafka.common.errors.UnknownTopicOrPartitionException — the Kafka-protocol equivalent when a topic does not exist, seen via the broker rather than directly from ZooKeeper.
• kafka.common.InconsistentClusterIdException — the broker’s stored cluster ID does not match the one in ZooKeeper, often after pointing a broker at a different ensemble.

Frequently Asked Questions

Does this error mean I lost my Kafka data? Almost never. NoNode is a path lookup miss, not data corruption. In the vast majority of cases you are reading the wrong chroot or the wrong ensemble. Verify the location first with ls / before assuming any loss.

How do I tell if my cluster uses a chroot? Check zookeeper.connect in server.properties. If it ends in a path segment after the host:port list (e.g. .../kafka), that segment is the chroot and all Kafka znodes live under it. A bare host:2181 means the root is used.

Should I manually create the missing znode? No. Broker registrations under /brokers/ids are ephemeral and tied to a live session; a hand-created node is permanent and will cause NodeExists and controller issues. Fix the path you are reading, or restart the broker so it registers itself.

Why does only one of my tools hit this while brokers run fine? The brokers and that tool disagree on the connection string. The brokers use the chroot; the tool omits it (or targets a different ensemble). Align the tool’s connection string with the brokers’.

Does this happen on KRaft clusters? No. KRaft clusters have no ZooKeeper, so there are no /brokers/ids znodes and no NoNodeException. Metadata lives in the internal __cluster_metadata log, inspected with kafka-metadata-quorum.sh.