🛟 ZFS on Linux: Pools, Snapshots, and Scrubs for Data You Can't Lose

The first time ZFS told me a disk was silently corrupting data — not failed, not throwing SMART errors, just quietly returning wrong bytes — and then repaired the corruption from a redundant copy without me lifting a finger, I understood why people who run ZFS get evangelical about it. Most filesystems trust that what the disk hands back is what you wrote. ZFS checksums every block and assumes the hardware is a liar until proven honest. On data you genuinely cannot afford to lose, that assumption is the whole point.

ZFS on Linux (now OpenZFS) is mature and widely deployed, but it has its own vocabulary and its own failure modes, and treating it like a faster ext4 leads to grief. Here’s how I actually run ZFS pools, snapshots, and the scrubs that make the self-healing possible — and where an AI copilot helps decode a zpool status that’s trying to tell me something.

The pool is the unit, redundancy is mandatory

ZFS combines volume management and the filesystem into one layer. You build a pool (zpool) out of vdevs (virtual devices), and datasets live on the pool. The critical rule that newcomers miss: ZFS can only self-heal if there’s redundancy in the vdev. A single-disk pool checksums data and will tell you about corruption — but it can’t fix it, because there’s no second copy to heal from.

# A mirror of two disks — ZFS can detect AND repair silent corruption
sudo zpool create tank mirror /dev/disk/by-id/wwn-0x... /dev/disk/by-id/wwn-0x...

# Confirm
zpool status tank

Always reference disks by /dev/disk/by-id/ (stable WWN/serial paths), never /dev/sdX — those names reshuffle across reboots and you do not want a pool’s identity tied to enumeration order.

Datasets and properties

Datasets are like mountable, individually-tunable filesystems within the pool:

sudo zfs create tank/srv
sudo zfs create tank/home

# Per-dataset properties: compression is nearly always a win
sudo zfs set compression=lz4 tank/srv
sudo zfs set atime=off tank/srv        # skip access-time writes

compression=lz4 is essentially free CPU for real space savings on most data — turn it on. atime=off avoids a write on every read.

Snapshots: instant, atomic, cheap

ZFS snapshots are atomic, read-only, point-in-time captures of a dataset that cost nothing until data diverges:

# Snapshot before a risky change
sudo zfs snapshot tank/srv@pre-upgrade-2026-06-22

# List snapshots
zfs list -t snapshot

# Browse the snapshot without restoring (it's right there)
ls /tank/srv/.zfs/snapshot/pre-upgrade-2026-06-22/

# Roll the dataset back (destroys changes since the snapshot!)
sudo zfs rollback tank/srv@pre-upgrade-2026-06-22

The .zfs/snapshot/ directory is a quiet superpower: you can copy individual files straight out of a snapshot without a full rollback. For replication, zfs send/zfs receive streams snapshots to another pool or host — the basis of real off-site ZFS backups.

Scrubs: the chore that makes self-healing real

A scrub reads every block in the pool, verifies its checksum, and repairs any mismatch from a redundant copy. This is how ZFS catches bit-rot before you hit the bad block in production. Scrubs are not optional maintenance — they’re the mechanism that turns “I have redundancy” into “my data is actually intact.”

sudo zpool scrub tank
zpool status tank        # watch progress and results

Schedule scrubs regularly (monthly is a common cadence) via a systemd timer. Most distro ZFS packages ship one — confirm it’s enabled:

systemctl list-timers | grep zfs
# e.g. zfs-scrub-monthly@tank.timer

Reading zpool status — and where AI helps

zpool status is the dashboard, and under stress it can be dense: a degraded vdev, checksum-error counts, a resilver in progress, and cryptic per-device states all at once. Decoding it correctly — especially distinguishing “a disk truly failed” from “ZFS caught and fixed transient errors” — is exactly where I lean on an AI copilot.

Prompt: You are a senior ZFS storage engineer. Here is my zpool status -v tank output showing one device in DEGRADED state with CKSUM errors and a resilver at 60%. Tell me whether this disk is failing or whether ZFS already repaired the data, what the resilver is doing, whether my data is currently at risk, and the exact next steps — including what NOT to do mid-resilver.

Response (excerpt): The CKSUM (not READ/WRITE) errors mean ZFS detected checksum mismatches and, because this is a mirror, repaired them from the good copy — your data is intact. DEGRADED + an active resilver means ZFS is rebuilding redundancy onto the affected/replacement device; until it hits 100%, you’re running without full redundancy, so a second-disk failure now WOULD lose data. Do NOT reboot, export the pool, or pull a disk mid-resilver — let it finish. After it completes, check SMART (smartctl -a) on the CKSUM-erroring device: persistent checksum errors with no SMART faults can indicate cabling/controller issues, not just the disk. Then zpool clear tank to reset the counters and re-scrub to confirm zero new errors.

That’s the model doing real interpretive work: it distinguished CKSUM errors (caught-and-healed) from READ/WRITE failures, explained that I’m temporarily without redundancy during the resilver, and — most importantly — told me what not to do (no reboot, no disk pull mid-resilver, a classic way to turn a recoverable situation into data loss). I then verified with smartctl myself before deciding whether to replace the drive. AI decodes the status and the safe sequence; the human watches the resilver finish and confirms the hardware verdict before acting.

The traps worth respecting

• No redundancy, no healing. A single-disk or striped pool detects corruption but can’t fix it. Mirror or raidz if the data matters.
• ARC eats RAM by design. ZFS’s cache (ARC) will use a large share of RAM; that’s normal and returned under pressure, but size hosts accordingly.
• Snapshots aren’t backups. Same pool, same disks. Use zfs send to another machine for real backup.
• Never rm -rf a snapshot dir to free space. Destroy snapshots with zfs destroy; the .zfs directory isn’t a normal filesystem you can delete from.

The takeaway

ZFS gives Linux admins something most filesystems don’t: end-to-end checksumming that catches silent corruption and, with redundancy, repairs it automatically — plus atomic snapshots and replication on top. The non-negotiables are real redundancy (no healing without it), regular scrubs (the mechanism that finds rot before you do), and the discipline to remember snapshots live on the same disks and aren’t backups. AI is a genuinely strong partner for decoding a tense zpool status and laying out the safe recovery sequence, especially the “don’t touch it mid-resilver” guardrails. But the model reads the dashboard; you watch the resilver complete, confirm the hardware with SMART, and stay in control of any disk replacement. ZFS distrusts the hardware so you can trust the data — your job is to keep the scrubs running and the human in the loop on recovery.

For the broader storage decision, see choosing a Linux filesystem and managing software RAID with mdadm for the non-ZFS redundancy path. The smartctl disk health pre-failure triage prompt pairs well with ZFS scrub results when a device starts erroring.