🚨 AWS Error: 'ProvisionedThroughputExceededException' on Kinesis — Cause, Fix, and Troubleshooting Guide

Overview

Kinesis Data Streams enforces hard per-shard limits: writes are capped at 1 MB/s or 1,000 records/s, and reads at 2 MB/s across a maximum of 5 GetRecords calls per second. Exceed any of these on a shard and Kinesis rejects the call with ProvisionedThroughputExceededException. Because the limit is per-shard, a stream with plenty of aggregate capacity can still throttle if one shard is hot.

You will see it from a producer or consumer:

An error occurred (ProvisionedThroughputExceededException) when calling the PutRecord operation: Rate exceeded for shard shardId-000000000003 in stream events-stream under account 111122223333.

It occurs when producers push a shard past 1 MB/s or 1,000 rec/s (often due to a skewed partition key), or consumers poll a shard past its read limits.

Symptoms

• PutRecord/PutRecords intermittently fails with ProvisionedThroughputExceededException; PutRecords returns partial failures with this code per-record.
• Consumers (KCL/Lambda) log throttling and fall behind; iterator age climbs.
• CloudWatch shows WriteProvisionedThroughputExceeded / ReadProvisionedThroughputExceeded > 0 on specific shards.
• Load is uneven: total stream throughput is well under capacity but one shard is saturated.

Common Root Causes

1. A skewed partition key (hot shard)

Most records share a partition key (e.g. a single tenant ID or a constant), so they all hash to one shard which saturates while others idle.

2. Too few shards for the aggregate rate

Total ingest genuinely exceeds shards × 1 MB/s (or × 1,000 rec/s).

3. Unbatched single-record writes

High-frequency PutRecord calls hit the 1,000 rec/s per-shard cap long before the 1 MB/s byte cap.

4. Too many consumers polling one shard

More than 5 GetRecords/s per shard (multiple classic consumers without enhanced fan-out) trips the read limit.

5. No backoff on producers

Producers that don’t retry with backoff turn transient throttles into sustained ones.

How to diagnose

Step 1: Check the stream’s shard count and mode

aws kinesis describe-stream-summary --stream-name events-stream \
  --query 'StreamDescriptionSummary.[StreamStatus,OpenShardCount,StreamModeDetails.StreamMode]' \
  --output text

Step 2: Find which shard is hot in CloudWatch

aws cloudwatch get-metric-statistics --namespace AWS/Kinesis \
  --metric-name WriteProvisionedThroughputExceeded \
  --dimensions Name=StreamName,Value=events-stream \
  --start-time "$(date -u -d '30 min ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 60 --statistics Sum --output table

A single shard dominating the sum means skew, not aggregate overload.

Step 3: Inspect partition-key distribution

Check your producer: a constant or low-cardinality PartitionKey is the classic hot-shard cause. High-cardinality keys (per-event UUID) spread evenly.

Fixes

Spread load with a high-cardinality partition key

Use a key with many distinct values (user ID, request ID) so records hash across all shards instead of piling onto one.

Add shards (or switch to on-demand)

# Provisioned mode: raise the shard count
aws kinesis update-shard-count --stream-name events-stream \
  --target-shard-count 8 --scaling-type UNIFORM_SCALING

# Or remove capacity planning entirely
aws kinesis update-stream-mode --stream-arn arn:aws:kinesis:us-east-1:111122223333:stream/events-stream \
  --stream-mode-details StreamMode=ON_DEMAND

Batch writes and add backoff

Use PutRecords (up to 500 records/call) and retry only the throttled entries with exponential backoff instead of hot-looping single PutRecord calls.

Use enhanced fan-out for many consumers

Register consumers with enhanced fan-out (dedicated 2 MB/s per shard each) instead of sharing the 5 GetRecords/s read budget.

What to watch out for

• Limits are per shard: aggregate headroom doesn’t help a hot shard — fix the partition key first, add shards second.
• PutRecords succeeds at the request level while individual records fail; always inspect FailedRecordCount and retry only failures.
• update-shard-count is rate-limited and resharding is not instant; on-demand mode auto-scales but costs more per GB.
• The 1,000 records/s cap bites high-frequency tiny writes before the 1 MB/s byte cap — batch aggressively.

Related

• AWS Error: DynamoDB ‘ProvisionedThroughputExceededException’ — the same code on a different service, with hot-partition parallels.
• AWS Error: ‘Throttling: Rate exceeded’ / RequestLimitExceeded — control-plane API throttling and backoff patterns.
• AWS Error: Lambda ‘TooManyRequestsException: Rate Exceeded’ — throttling on the consumer side.