🚨 Kafka Error Guide: 'InvalidPartitionsException' Cannot Decrease Partition Count

Exact Error Message

This error surfaces when you run a topic alter operation that asks Kafka for fewer partitions than the topic already has. The CLI wraps it as an ExecutionException, but the root cause is the broker-side InvalidPartitionsException:

Error while executing topic command : Topic currently has 6 partitions, which is higher than the requested 3.
[2026-06-29 14:02:11,884] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.InvalidPartitionsException: Topic currently has 6 partitions, which is higher than the requested 3.
	at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
	at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
	at kafka.admin.TopicCommand$AdminClientTopicService.alterTopic(TopicCommand.scala:404)
	at kafka.admin.TopicCommand$.main(TopicCommand.scala:66)
Caused by: org.apache.kafka.common.errors.InvalidPartitionsException: Topic currently has 6 partitions, which is higher than the requested 3.
 (kafka.admin.TopicCommand$)

You may also see the variant The partition count for a topic can only be increased or, for a count of zero, Number of partitions must be larger than 0.

What the Error Means

Kafka partition counts are monotonic — they can only go up, never down. The number of partitions defines the parallelism and the key-to-partition mapping for a topic, and Kafka has no safe online operation to merge or remove partitions. When you call kafka-topics.sh --alter --partitions N with an N that is less than the current partition count, the controller rejects the request with InvalidPartitionsException rather than silently doing nothing.

The same exception covers a few related invalid inputs:

• Requesting --partitions 0 or a negative value (partitions must be a positive integer).
• Requesting a count equal to or below the current count during an alter (you must request strictly more).
• Infrastructure-as-code drift where a tool re-applies a lower desired partition count than what the live topic already grew to.

The key takeaway: this is not a transient or connectivity failure. It is Kafka enforcing an invariant, and no retry will make a shrink succeed.

Common Causes

1. Trying to reduce partitions with --alter

The most direct cause: someone (or a pipeline) ran --alter --partitions 3 against a topic that already has 6. Kafka refuses because shrinking would orphan data and break the key hashing contract.

2. Requesting zero or a negative partition count

A templated command or environment variable that evaluates to 0 (or an empty value coerced to 0) produces Number of partitions must be larger than 0, a sibling of the same error family.

3. IaC / GitOps drift setting a lower count

Terraform, Strimzi KafkaTopic resources, or a custom operator may hold a desired partitions = 3 while an operator manually bumped the live topic to 6 for throughput. On the next reconcile, the tool tries to “correct” the count downward and the broker rejects it. The state file and the live cluster have diverged.

4. Re-running a create-or-alter script after a manual increase

Idempotent provisioning scripts that “ensure” a partition count can break when the live value was increased out of band, because the script’s hardcoded count is now lower than reality.

How to Reproduce the Error

Create a topic with 6 partitions, then attempt to shrink it:

kafka-topics.sh --bootstrap-server localhost:9092 \
  --create --topic orders --partitions 6 --replication-factor 1

# This alter will FAIL with InvalidPartitionsException
kafka-topics.sh --bootstrap-server localhost:9092 \
  --alter --topic orders --partitions 3

The alter immediately returns Topic currently has 6 partitions, which is higher than the requested 3. You can also reproduce the zero-count variant by passing --partitions 0 on either create or alter.

Diagnostic Commands

All commands here are read-only — they inspect state without changing it.

Confirm the current partition count for the topic:

kafka-topics.sh --bootstrap-server localhost:9092 \
  --describe --topic orders

Topic: orders	TopicId: 9aF2... PartitionCount: 6	ReplicationFactor: 1	Configs: ...
	Topic: orders	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	...
	Topic: orders	Partition: 5	Leader: 1	Replicas: 1	Isr: 1

The PartitionCount: 6 is the number you cannot go below. List all topics to spot duplicates or naming mistakes:

kafka-topics.sh --bootstrap-server localhost:9092 --list

Inspect any topic-level overrides (retention, cleanup policy) before you consider a recreate:

kafka-configs.sh --bootstrap-server localhost:9092 \
  --describe --entity-type topics --entity-name orders

Verify broker reachability and supported API versions if the command itself is failing oddly:

kafka-broker-api-versions.sh --bootstrap-server localhost:9092 | head -5

Check the controller/broker log for the rejected alter to confirm the source request:

journalctl -u kafka --since "10 min ago" | grep -i InvalidPartitions

Step-by-Step Resolution

There are two distinct situations: you actually wanted to grow the topic (and got the error from a stale lower number), or you genuinely need fewer partitions.

Case A: You meant to increase partitions

If your intent was to add parallelism, simply request a number strictly greater than the current count. To go from 6 to 12:

kafka-topics.sh --bootstrap-server localhost:9092 \
  --alter --topic orders --partitions 12

This succeeds and is an online operation. Important caveat: increasing partitions changes the partition assignment for keyed messages. Kafka maps a key with hash(key) % partitionCount, so adding partitions sends future records for a given key to a different partition than past records. That breaks per-key ordering guarantees and any consumer logic that assumes a key always lands in one partition. Plan the increase during a quiet window and confirm downstream consumers tolerate the re-mapping.

Case B: You genuinely need fewer partitions

Kafka cannot shrink a topic in place. The only way to reduce the partition count is to recreate the topic, which means accepting data loss or reprocessing from the source:

1. Stop or pause producers writing to the topic.

2. Let consumers drain, or capture offsets if you plan to reprocess.

3. Delete the topic:

   kafka-topics.sh --bootstrap-server localhost:9092 \
     --delete --topic orders

4. Recreate it with the lower count and the same configs:

   kafka-topics.sh --bootstrap-server localhost:9092 \
     --create --topic orders --partitions 3 --replication-factor 1 \
     --config retention.ms=604800000

5. Restart producers and reset/re-seed data as needed (replay from a source of truth or upstream topic).

Treat this as data-destructive; never do it casually on a topic with unconsumed business data.

Case C: Fixing IaC drift

If the error came from a reconcile loop, the fix is to align desired state with reality rather than letting the tool drive the count down. Update the Terraform/Strimzi manifest to the actual live partition count (e.g. set partitions = 6), then re-apply. Never let automation attempt a downward change — most providers will surface this exact exception.

Prevention and Best Practices

• Size partitions generously up front. Because you can only grow, choose a partition count that covers projected peak throughput plus headroom, so you rarely need to alter at all.
• Treat partition count as immutable in IaC. Pin the desired value to the live value and require a deliberate, reviewed change to increase it. Guard against any code path that computes a count dynamically and could produce a lower number.
• Validate inputs in provisioning scripts. Reject 0, negatives, and any requested count below the current PartitionCount before calling the admin API.
• Document the ordering trade-off. Make it a checklist item that increasing partitions re-maps keys, so teams do not silently break ordering-sensitive pipelines.
• Use a single source of truth for topic config. Avoid mixing manual --alter commands with a reconciler; one or the other should own partition counts.
• For fast triage of a partition or topic error during an incident, the free incident assistant can turn the CLI output into a likely cause.

Related Errors

• org.apache.kafka.common.errors.InvalidReplicationFactorException — requesting a replication factor higher than the number of available brokers (a separate invariant, but a common companion when scripting topic creation).
• TopicExistsException — re-running a create against an existing topic; often masks an intended alter.
• UnknownTopicOrPartitionException — addressing a topic or partition that does not exist, sometimes seen right after a delete-and-recreate before metadata propagates.
• org.apache.kafka.common.errors.PolicyViolationException — a create.topic.policy plugin rejecting the partition or replication settings.

Frequently Asked Questions

Can I ever decrease the number of partitions on a Kafka topic?

Not in place. Kafka only supports increasing partitions. To end up with fewer, you must delete and recreate the topic (or write to a brand-new topic with the lower count and migrate consumers), accepting data loss or reprocessing.

Why does Kafka allow increasing but not decreasing partitions?

Adding partitions is an append-only metadata change. Removing them would require merging or discarding existing partition logs and would silently break the hash(key) % partitionCount mapping for all existing keyed data — there is no safe, general way to do that, so Kafka forbids it.

Will increasing partitions break message ordering?

It can. Kafka guarantees ordering only within a partition. Because adding partitions changes which partition a given key maps to, records for the same key may land in a different partition after the change, breaking per-key ordering for downstream consumers. Plan increases carefully.

I only got this error from Terraform — is my topic broken?

No. The topic is fine; the reconciler simply tried to set a count lower than the live value. Update your manifest to match the actual PartitionCount (or a higher target) and re-apply. The exception is the broker protecting the topic, not a sign of corruption.