🚨 Kafka Error Guide: 'UnknownTopicOrPartitionException' Topic Not Found

Exact Error Message

This exception shows up when a producer, consumer, or admin client references a topic-partition the broker does not have:

[2026-06-29 11:47:02,118] WARN [Consumer clientId=consumer-1, groupId=billing] Error while fetching metadata with correlation id 12 : {payments-v3=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

It is often reported as Partition not found or surfaced as a producer failure:

org.apache.kafka.common.errors.UnknownTopicOrPartitionException:
  This server does not host this topic-partition.
[2026-06-29 11:47:05,420] ERROR Error when sending message to topic payments-v3 with key: null, value: 240 bytes (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

What the Error Means

The broker received a request for a specific topic-partition and replied that it does not host it. There are two distinct meanings depending on context:

1. The topic genuinely does not exist anywhere in the cluster (never created, deleted, or a typo).
2. The topic exists, but the contacted broker is not a leader/replica for that partition, or the client is using stale metadata that points at the wrong broker.

It is a retriable exception, which is why transient cases (stale metadata after a reassignment, or a topic still being created) clear on their own after a metadata refresh. The non-transient cases — a misspelled topic name or auto-creation disabled for a topic the producer assumed existed — repeat forever until corrected.

The defining signal is whether the topic shows up in a cluster-wide describe. If kafka-topics.sh --list does not include it, the topic truly does not exist.

Common Causes

• Topic does not exist. It was never created, or a deploy assumed an external process would create it.
• Auto-create disabled. With auto.create.topics.enable=false (the production-safe setting), producing to a non-existent topic fails instead of silently creating it.
• Typo or wrong name. A casing or character mismatch between client config and the actual topic.
• Topic was deleted while clients still reference it.
• Stale client metadata after a partition reassignment or leader move points the client at a broker that no longer hosts the partition.
• Partition number out of range. The client targets partition 12 of a 6-partition topic.
• Wrong cluster. bootstrap.servers points at a different environment’s cluster that lacks the topic.

How to Reproduce the Error

With auto-create disabled, produce to a topic that does not exist:

# Confirm the topic is absent
kafka-topics.sh --bootstrap-server kafka-1:9092 --list | grep payments-v3 || echo "not present"
# A producer/consumer targeting payments-v3 now logs UNKNOWN_TOPIC_OR_PARTITION
kafka-console-consumer.sh --bootstrap-server kafka-1:9092 --topic payments-v3 --from-beginning

You can also reproduce the stale-metadata variant by reassigning a partition’s replicas and immediately issuing a request with a client holding the pre-reassignment metadata; the request races to the old broker and is rejected until the client refreshes.

Diagnostic Commands

Confirm whether the topic exists cluster-wide and inspect its partitions:

kafka-topics.sh --bootstrap-server kafka-1:9092 --list | grep -i payments
kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --topic payments-v3

Check the broker’s auto-create setting and verify the target cluster identity:

grep -iE 'auto.create.topics.enable|num.partitions' /opt/kafka/config/server.properties
kafka-cluster.sh cluster-id --bootstrap-server kafka-1:9092

For consumer groups, see what topic/partitions the group is actually bound to:

kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --describe --group billing
kafka-broker-api-versions.sh --bootstrap-server kafka-1:9092
sudo journalctl -u kafka --since "10 min ago" | grep -iE 'UNKNOWN_TOPIC|delete|created topic'

Step-by-Step Resolution

1. Check whether the topic exists. kafka-topics.sh --list | grep <name>. Mind exact casing. If it is missing, that is your answer.
2. Verify you are on the right cluster. kafka-cluster.sh cluster-id and compare with the cluster you expect. A wrong bootstrap.servers is a common silent cause.
3. If the topic should exist but does not, create it (deliberately, with the right partitions and replication factor) or fix the upstream provisioning that was supposed to.
4. If it exists, describe it. Confirm the partition number your client targets is within range and that partitions have leaders.
5. Rule out stale metadata. If the topic and partitions look healthy but the client still errors, force a client restart or wait for metadata.max.age.ms so it refetches metadata; transient post-reassignment cases clear here.
6. Fix the name. If it is a typo, correct the client’s topic configuration rather than creating a mistakenly named topic.
7. Confirm recovery. Re-run the consumer/producer; the warning should stop once the topic exists and metadata is current.

Prevention and Best Practices

• Keep auto.create.topics.enable=false in production and provision topics explicitly through infrastructure-as-code so missing topics fail loudly at deploy time, not silently at runtime.
• Validate topic names in client config (a shared constant or schema) to eliminate typos and casing drift.
• Pin clients to environment-specific bootstrap.servers and assert the expected cluster id at startup to catch wrong-cluster mistakes.
• Treat the exception as retriable in code so transient stale-metadata cases recover automatically.
• After partition reassignments, expect brief UNKNOWN_TOPIC_OR_PARTITION noise as clients refresh; alert only when it persists.
• Document the topic-creation workflow so teams do not assume topics appear on first produce. More patterns are in the Kafka guides.

Related Errors

• LeaderNotAvailableException — the topic exists but a partition has no leader yet (often right after creation).
• NotLeaderOrFollowerException — the topic-partition exists, but this broker is not its leader; metadata is stale.
• TimeoutException: Topic ... not present in metadata — the downstream timeout when the topic never appears in metadata.
• InvalidTopicException — the topic name itself is invalid (illegal characters or length).

Frequently Asked Questions

Does this mean my topic was deleted? Not necessarily. It means the contacted broker does not host the topic-partition — because it does not exist, was deleted, or the broker is not its replica. Run kafka-topics.sh --list to see if the topic exists at all.

Why did it work in dev but not prod? Dev often has auto.create.topics.enable=true, which silently creates topics on first produce. Production usually disables that, so the same code fails with UnknownTopicOrPartitionException.

It appears briefly after a reassignment, then stops. Is that a bug? No. That is expected transient behavior while clients refresh stale metadata after partitions move. It is only a problem if it persists.

Could a wrong partition number cause this? Yes. Targeting a partition index beyond the topic’s partition count produces this exception. Check the count with kafka-topics.sh --describe.

Is it safe to just retry? For transient (metadata) cases, yes — it is retriable. For a missing or misnamed topic, retries will never succeed; you must create the topic or fix the name.