🚨 Redis Error Guide: 'Bad file format reading the append only file' / Short Read Loading DB

Overview

When Redis starts, it loads its dataset from disk — either the AOF (append-only file) or the RDB snapshot. If that file is truncated, corrupted, or too large to fit in memory, Redis refuses to start and exits with a fatal error rather than come up with partial data. The most common messages are a “Bad file format” / “Short read” while loading the AOF, or a checksum/short-read failure loading the RDB, or an out-of-memory abort partway through the load.

The literal errors from the Redis log:

# Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>

# Short read or OOM loading DB. Unrecoverable error, aborting now.

# Wrong signature trying to load DB from file
# Internal error in RDB reading offset 0, function at rdb.c:...

A common trigger is an unclean shutdown (power loss, OOM-kill, kill -9) that leaves the AOF’s last command half-written. Because Redis validates the file, it stops instead of silently dropping data — which means recovery is a deliberate, backup-first process.

Symptoms

• redis-server exits immediately on start; the service is in a restart loop.
• Log contains “Bad file format reading the append only file” or “Short read or OOM loading DB”.
• The crash followed a host power loss, OOM-kill, disk-full event, or kill -9.
• redis-check-aof/redis-check-rdb report truncation at a specific offset.

journalctl -u redis --no-pager | tail -15

# Bad file format reading the append only file
# Short read or OOM loading DB. Unrecoverable error, aborting now.

Common Root Causes

1. Truncated AOF from an unclean shutdown

Power loss or kill -9 mid-write leaves the last AOF command incomplete. This is the classic “Short read” case and is usually fully recoverable.

redis-cli CONFIG GET dir
ls -lh $(redis-cli CONFIG GET dir | tail -1)/appendonlydir 2>/dev/null || \
ls -lh $(redis-cli CONFIG GET dir | tail -1)/*.aof 2>/dev/null

2. Corrupted RDB (bad checksum / wrong signature)

A partial copy, disk error, or interrupted BGSAVE produces an RDB that fails its signature/CRC check.

# Wrong signature trying to load DB from file

3. Disk full during a write/rewrite

If the volume filled while AOF/RDB was being written, the file is truncated or zero-length.

df -h $(redis-cli CONFIG GET dir | tail -1)

4. Dataset too large for available memory (OOM on load)

The persisted data exceeds RAM (or maxmemory plus overhead), so the load aborts with OOM.

grep -i 'killed process' /var/log/syslog 2>/dev/null | tail
free -h

Diagnostic Workflow

Always work on a copy. Back up the persistence files before running any --fix.

Step 1: Read the exact fatal message

journalctl -u redis --no-pager | grep -iE 'append only file|Short read|signature|Unrecoverable|OOM' | tail

This tells you AOF vs. RDB and truncation vs. corruption vs. OOM.

Step 2: Locate and back up the files

DIR=$(redis-cli CONFIG GET dir 2>/dev/null | tail -1); echo "$DIR"
# Redis 7+ multi-part AOF lives in appendonlydir/
cp -a "$DIR/appendonlydir" "$DIR/appendonlydir.bak.$(date +%s)" 2>/dev/null
cp -a "$DIR"/dump.rdb "$DIR"/dump.rdb.bak.$(date +%s) 2>/dev/null

Step 3: Check the AOF (truncation is recoverable)

# Redis 7 multi-part AOF: check the manifest + incr file inside appendonlydir/
redis-check-aof "$DIR/appendonlydir/appendonly.aof.1.incr.aof"
# If it reports truncation, fix a COPY (trims the last partial command):
redis-check-aof --fix "$DIR/appendonlydir/appendonly.aof.1.incr.aof"

0x            a1b2: Expected \r\n, got: 2f2f
AOF analyzed: ... last valid offset 41232
This will shrink the AOF ... Continue? [y/N]

Alternatively, set aof-load-truncated yes so Redis tolerates a truncated tail on load.

# redis.conf
appendonly yes
aof-load-truncated yes

Step 4: Check the RDB

redis-check-rdb "$DIR/dump.rdb"

[offset 0] Checking RDB file dump.rdb
--- RDB ERROR DETECTED ---

redis-check-rdb cannot repair, but it tells you where corruption starts and whether the file is salvageable. If AOF is enabled and healthy, prefer recovering from AOF (or from a replica/backup).

Example Root Cause Analysis

After a host lost power, redis-server entered a crash loop. The log:

# Bad file format reading the append only file: make a backup of your AOF file, then use ./redis-check-aof --fix <filename>

This is the truncated-AOF signature — the last write never completed. Following the safe procedure, ops backed up appendonlydir/, then checked the incremental AOF:

redis-check-aof appendonlydir/appendonly.aof.1.incr.aof

AOF analyzed: filename=..., size=..., last valid offset=41232
0x a1b2: Expected \r\n, got: 2f2f

Only the final partial command was bad. Running --fix on the copy trimmed that last incomplete entry:

redis-check-aof --fix appendonlydir/appendonly.aof.1.incr.aof

Successfully truncated AOF

After restoring the fixed file, Redis started cleanly and lost only the single un-acknowledged write. To prevent recurrence, aof-load-truncated yes was confirmed (default) so a truncated tail auto-recovers, and the host got a UPS-backed shutdown. Total data loss: one in-flight command.

Prevention Best Practices

• Keep aof-load-truncated yes (default) so an unclean-shutdown tail truncation auto-recovers without manual repair.
• Use appendfsync everysec for a good durability/latency balance; understand always is safest, no least safe.
• Always back up persistence files before any --fix; run repairs on copies, never the original.
• Run a replica and/or scheduled RDB backups so a corrupt file on one node is never your only copy — restore from a healthy source instead of repairing.
• Monitor disk free on the dir volume; a full disk during rewrite is a top cause of truncation.
• Size memory so the dataset loads without OOM; keep vm.overcommit_memory=1. See more Redis error guides.

Quick Command Reference

# Exact fatal message
journalctl -u redis | grep -iE 'append only file|Short read|signature|Unrecoverable|OOM' | tail

# Locate + BACK UP first
DIR=$(redis-cli CONFIG GET dir | tail -1)
cp -a "$DIR/appendonlydir" "$DIR/appendonlydir.bak.$(date +%s)"
cp -a "$DIR"/dump.rdb "$DIR"/dump.rdb.bak.$(date +%s)

# Check / repair AOF (on a copy)
redis-check-aof "$DIR/appendonlydir/appendonly.aof.1.incr.aof"
redis-check-aof --fix "$DIR/appendonlydir/appendonly.aof.1.incr.aof"

# Check RDB
redis-check-rdb "$DIR/dump.rdb"

# Disk / memory sanity
df -h "$DIR"
free -h

Conclusion

“Bad file format reading the append only file” and “Short read or OOM loading DB” mean Redis refused to start because its persistence file is truncated, corrupted, or too big for memory — it stops rather than load partial data. The causes:

1. A truncated AOF from an unclean shutdown (the most common, and recoverable).
2. A corrupted RDB failing its signature/checksum.
3. A disk-full event truncating the file mid-write.
4. A dataset too large for RAM, aborting the load with OOM.

Read the log to classify AOF vs. RDB, back up first, then use redis-check-aof --fix (truncation) or redis-check-rdb (diagnosis) on copies — or better, restore from a replica/backup. Keep aof-load-truncated yes, watch disk free, and never let a single file be your only copy.