🚨 Kafka Error Guide: 'InvalidRecordException: One or more records have been rejected' Records Rejected by Broker Validation

Exact Error Message

The producer-facing exception is short and unhelpful on its own. The real reason lives in the broker logs.

org.apache.kafka.common.InvalidRecordException: One or more records have been rejected
	at org.apache.kafka.clients.producer.internals.ProducerBatch.completeExceptionally(ProducerBatch.java:308)
	at org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:758)
	at org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:691)
	at org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:612)

# Broker-side server.log, compaction variant:
[2026-06-29 11:04:23,118] ERROR [ReplicaManager broker=2] Error processing append operation
  on partition events-cdc-7 (kafka.server.ReplicaManager)
org.apache.kafka.common.record.InvalidRecordException: Compacted topic cannot accept
  message without key in topic partition events-cdc-7

# Broker-side server.log, timestamp variant:
org.apache.kafka.common.errors.InvalidTimestampException: Timestamp 1714000000000 of
  message with offset 0 is out of range. The timestamp should be within
  [1751000000000, 1751086400000]

What the Error Means

InvalidRecordException is the broker telling the producer: “I parsed your record batch, ran it through record validation, and at least one record violated a rule, so I rejected the whole batch.” It is distinct from a serialization failure (which happens client-side before the bytes ever leave the JVM) and from a network failure (where the broker never answered). Here the broker answered, and the answer was no.

Kafka validates every incoming batch in LogValidator before the records are appended to the log. The check runs on the leader during the produce request, after the bytes are deserialized into a MemoryRecords structure but before the high-water mark advances. Because validation is per-batch, one offending record poisons every record shipped in the same batch.

The umbrella message “One or more records have been rejected” is what the client surfaces; the specific subclass or text in the broker log is what you must act on.

Common Causes

• Null key on a log-compacted topic. A topic with cleanup.policy=compact (or compact,delete) requires every record to carry a non-null key, because compaction keys retention on the key. A null-key record has nothing to compact against, so the broker rejects it outright with “Compacted topic cannot accept message without key.”
• Record timestamp outside the allowed window. When message.timestamp.type=CreateTime (the default), the broker compares the producer-supplied timestamp against the broker clock. If the difference exceeds message.timestamp.difference.max.ms, the record is rejected with InvalidTimestampException. This commonly bites backfills replaying historical data with old CreateTime values, or producers with a badly skewed clock.
• Magic byte / format mismatch or corruption. A malformed batch, an incompatible record format version, or a corrupted CRC produces a broker-side record validation failure. (A purely corrupted-on-the-wire batch may instead surface as CorruptRecordException.)
• Transactional records sent to a non-transactional context, or records with transactional markers the broker cannot honor, fail batch validation.

How to Reproduce the Error

The compaction variant is the easiest to trigger deterministically. Produce a null-key record to a compacted topic.

// Topic created with cleanup.policy=compact
Properties p = new Properties();
p.put("bootstrap.servers", "broker:9092");
p.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
p.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

try (Producer<String, String> producer = new KafkaProducer<>(p)) {
    // key == null on a compacted topic -> InvalidRecordException
    producer.send(new ProducerRecord<>("events-cdc", null, "{\"id\":42}")).get();
}

For the timestamp variant, set a CreateTime far in the past while message.timestamp.difference.max.ms is small:

long ancient = System.currentTimeMillis() - Duration.ofDays(30).toMillis();
producer.send(new ProducerRecord<>("metrics", null, ancient, "k", "v")).get();

Diagnostic Commands

Confirm whether the target topic is compacted and inspect its timestamp settings. These are read-only.

# Is this topic compacted? Look for cleanup.policy=compact
kafka-topics.sh --bootstrap-server broker:9092 \
  --describe --topic events-cdc

# Inspect timestamp + compaction configs explicitly
kafka-configs.sh --bootstrap-server broker:9092 \
  --entity-type topics --entity-name events-cdc --describe

# Pull the exact broker-side rejection reason
journalctl -u kafka --since "10 min ago" | grep -i "InvalidRecord\|InvalidTimestamp\|Compacted topic"

# Search rolled broker logs for the validation failure
grep -iE "Compacted topic cannot accept|Timestamp .* is out of range" /var/log/kafka/server.log

Watch for cleanup.policy=compact, the values of message.timestamp.type and message.timestamp.difference.max.ms, and the precise broker error string.

Step-by-Step Resolution

1. Read the broker log, not the client stacktrace. The client only says “rejected.” journalctl -u kafka or server.log carries the actionable subclass. Decide which of the three failure families you are in.

2. For the null-key-on-compacted-topic case, the fix is in the producer: every record must have a meaningful, stable key. If a stream genuinely has no natural key, the topic should not be compacted; it should use cleanup.policy=delete. Choosing the right cleanup policy is a topic-design decision, not a producer retry knob.

3. For the timestamp case, decide whether the producer clock or the policy is wrong. If you are intentionally backfilling old data, the cleanest answer is to let the broker stamp records on arrival by setting the topic to log-append time:

   message.timestamp.type=LogAppendTime

   If you must preserve original CreateTime values, widen the tolerance:

   message.timestamp.type=CreateTime
   message.timestamp.difference.max.ms=31536000000   # 365 days, for historical replays

   If neither applies, the real defect is clock skew on the producer host: fix NTP.

4. For magic-byte / format issues, align client and broker versions and confirm log.message.format.version (older clusters) is compatible with the client. Genuinely corrupted batches point at memory or disk problems on the producer side.

5. Re-test with the reproduction snippet and confirm the broker log no longer shows the rejection.

Prevention and Best Practices

• Make compaction a deliberate choice. Document, per topic, whether it is compact or delete, and enforce that every producer to a compacted topic emits non-null keys (validate in a serializer or interceptor).
• Prefer LogAppendTime for any ingest path that replays historical data; it sidesteps message.timestamp.difference.max.ms entirely.
• Keep producer hosts on disciplined NTP. Most “random” timestamp rejections are silent clock drift.
• Keep client and broker library versions in lockstep across your fleet to avoid record-format surprises.
• Because InvalidRecordException is not retriable, do not bury it under retries; surface it to a dead-letter path with the offending key and timestamp logged.
• For ongoing visibility into why batches are being rejected in production, an incident assistant can correlate the client exception with the broker-side reason automatically.

Related Errors

• SerializationException — fails client-side before the broker ever validates the batch; if you see that instead, the bytes never reached LogValidator.
• CorruptRecordException — a CRC/format failure detected on read or append; closely related to the magic-byte family of InvalidRecordException but signals on-the-wire or on-disk corruption rather than a policy violation.
• RecordTooLargeException — another broker-side rejection, but bounded by max.message.bytes rather than key/timestamp validation.
• InvalidRecordException’s transactional cousins surface when transactional markers are mishandled; see UnknownProducerIdException and OutOfOrderSequenceException.

More patterns are collected in the Kafka guides.

Frequently Asked Questions

Why does the producer only say “One or more records have been rejected” with no detail? The client deliberately surfaces a generic message because the authoritative reason is computed on the broker during append. Always pair the client stacktrace with the broker’s server.log (or journalctl -u kafka), where the concrete subclass — compaction, timestamp, or format — is logged.

Can I just turn on retries to get past InvalidRecordException? No. It is a non-retriable error: the same record will be rejected every time because it violates a deterministic validation rule. Retrying only wastes round trips. Fix the record (add a key, correct the timestamp) or the topic policy.

Why does one bad record fail my whole batch? Validation runs at batch granularity in LogValidator. If any record in the batch fails, the broker rejects the entire produce request, so well-formed records sent alongside the offender are also failed. Smaller batches reduce blast radius but do not change the rule.

My backfill of last month’s data keeps getting rejected — what’s the minimal fix? Either switch the topic’s message.timestamp.type to LogAppendTime so the broker stamps on arrival, or raise message.timestamp.difference.max.ms to cover your historical window. The former is simplest when you do not need to preserve original event times.

Does message.timestamp.difference.max.ms apply with LogAppendTime? No. Under LogAppendTime the broker overwrites the producer timestamp with its own clock, so the difference check is irrelevant. The setting only matters when message.timestamp.type=CreateTime.