Linux Error Guide: 'Read-only file system' (EROFS) — Recover After a Remount-RO and fsck
Fix EROFS 'Read-only file system' after ext4 remounts read-only on errors: diagnose with dmesg and tune2fs, verify disk health, run fsck safely, and remount rw.
- #linux
- #troubleshooting
- #errors
- #filesystem
Fixing errors like this? Get 500 free DevOps AI prompts
500 copy-paste AI prompts for the stack you actually run — one PDF, free.
Overview
You go to write a file and the shell throws it straight back at you:
bash: /var/log/app.log: Read-only file system
This is EROFS (errno 30). But the mount was read-write ten minutes ago, and nothing in /etc/fstab changed. What happened is that the kernel itself flipped the filesystem to read-only mid-flight.
Most ext4 filesystems are mounted with errors=remount-ro. When the ext4 driver hits a metadata inconsistency or an I/O error it cannot safely ignore, it stops trusting the on-disk structure and remounts the volume read-only on the spot. This is a protective reflex: continuing to write to a corrupt or failing filesystem risks turning a recoverable problem into unrecoverable data loss. So every write now returns EROFS until you intervene.
The critical thing to understand: the read-only state is a symptom, not the disease. The filesystem went read-only because the kernel found something wrong. The fix is to find and repair that underlying problem with fsck, not to blindly remount read-write and carry on.
Note: this guide is about a filesystem the kernel forced read-only after detecting errors. If your media is read-only because of a hardware write-protect switch or an inherently read-only device (SD card lock tab, read-only loop mount, write-protected USB), that is a different failure mode — see Linux Error: disk write-protected.
Symptoms
- Writes fail with
Read-only file system/EROFS, but reads still work. touch,apt, log rotation, database writes, and package installs all fail suddenly.mountshows the filesystem withroin its options even thoughfstabsaysrw(ordefaults).- Services crash or refuse to start because they cannot write PID files, sockets, or logs.
dmesgcontainsEXT4-fs errorlines followed byRemounting filesystem read-only.- On the root filesystem, the whole system becomes unstable — you may only be able to run commands already in cache.
Common Root Causes
- Filesystem metadata errors → remount-ro. ext4 detected a corrupt inode, bad block bitmap, or directory entry and, per
errors=remount-ro, protected itself by going read-only. - A failing or flaky underlying disk. Bad sectors, a dying SSD, or a loose SATA/NVMe connection cause I/O errors that the kernel surfaces as filesystem errors. This is the most important cause to rule out — repairing the filesystem on a dying disk just corrupts it again.
- Unclean shutdown. A power loss or hard reset left the journal or metadata in an inconsistent state that ext4 noticed on next access.
- Metadata corruption from other causes. Overlapping/misconfigured storage stacks, a bad RAID/LVM event, or previously ignored corruption that finally tripped a check.
Diagnostic Workflow
Confirm the read-only state, then find out why — before touching anything.
# 1. Which mounts are read-only right now?
mount | grep ' ro,'
# 2. Confirm the target, filesystem type, and current options
findmnt -o TARGET,FSTYPE,OPTIONS /var
findmnt -o TARGET,FSTYPE,OPTIONS / # for the root fs
# 3. What did the kernel say? Look for the ext4 error and the remount event
dmesg -T | grep -i 'ext4\|remounting\|read-only\|errors'
# 4. Full kernel log with timestamps (scroll to the first error)
journalctl -k -b
# 5. Inspect the filesystem's own error state (run against the block device)
tune2fs -l /dev/sda1 | grep -i 'state\|error'
In tune2fs -l output, watch two fields. Filesystem state: should read clean; after a remount-ro event it typically shows not clean with errors or clean with errors. FS Error count: will be non-zero, and First error time: / Last error time: tell you when the trouble started.
Check the disk hardware BEFORE you run fsck. If the drive is failing, fsck will either fail on the same bad sectors or “fix” the filesystem onto media that is about to die. Verify health first:
# Overall health verdict and error/reallocation counters
smartctl -H /dev/sda
smartctl -A /dev/sda | grep -i 'reallocated\|pending\|uncorrect\|crc'
# Confirm the kernel isn't logging low-level I/O errors
dmesg -T | grep -i 'i/o error\|medium error\|ata\|nvme'
If SMART reports FAILED, or you see rising Reallocated_Sector_Ct / Current_Pending_Sector / I/O errors in dmesg, stop. Image the disk with ddrescue and recover from a copy or from backups rather than running repairs on the original. For a deeper dive on interpreting these counters, see our hardware disk error guides.
Example Root Cause Analysis
A worker node’s application starts logging Read-only file system on a data volume mounted at /data (/dev/sdb1). The root filesystem is fine, so the box is still usable.
dmesg -T shows the smoking gun:
[Tue Jul 7 02:14:09 2026] EXT4-fs error (device sdb1): ext4_lookup:1855: inode #524291: comm app-writer: deleted inode referenced: 787too
[Tue Jul 7 02:14:09 2026] EXT4-fs (sdb1): Remounting filesystem read-only
smartctl -H /dev/sdb returns PASSED and the reallocation counters are zero, so the disk itself is healthy — this is metadata corruption, most likely from an earlier unclean shutdown. Safe to repair.
Because fsck must never run on a mounted filesystem, unmount it first, then repair:
# Stop anything using the mount, then unmount
systemctl stop app-writer.service
umount /data # use 'fuser -km /data' only if you must force off users
# Repair non-interactively (-y answers 'yes' to all fixes)
fsck -y /dev/sdb1
# Re-mount and confirm it's read-write again
mount /data
findmnt -o TARGET,OPTIONS /data
After fsck completes cleanly, tune2fs -l /dev/sdb1 | grep -i state should report clean and the error count resets. Restart the service and verify writes succeed.
Root filesystem case. You cannot unmount / on a running system, so you have two safe options:
-
Schedule a repair for the next boot. Create the flag file and reboot; the init/initramfs runs
fsckon/before it is mounted read-write:touch /forcefsck # if / is still writable enough; else set it from rescue rebootOn systemd systems you can instead pass
fsck.mode=forceon the kernel command line for one boot. (The legacy/forcefsckflag is honored by most distros and removed automatically after the check.) -
Boot to rescue / single-user or a live USB and run
fsckwhile/is unmounted or mounted read-only:# From a rescue shell where the root device is NOT mounted rw fsck -y /dev/sda1 mount -o remount,rw /
Do not “fix” a read-only root by only running mount -o remount,rw / and continuing. That clears the symptom while leaving the corruption in place — the kernel will very likely trip and remount read-only again, and each cycle can deepen the damage.
Prevention Best Practices
-
Keep
errors=remount-ro. It is the sane default.errors=continuehides corruption anderrors=panicreboots — remount-ro gives you a chance to intervene cleanly. -
Monitor the kernel log for ext4 errors. Alert on
EXT4-fs errorandRemounting filesystem read-onlyindmesg/journalctl -kso you catch the first event, not the outage. -
Watch disk health proactively. Run
smartctlchecks (orsmartd) and replace drives showing pending/reallocated sectors before they take a filesystem down. -
Ensure clean shutdowns. Use a UPS on physical hosts and let systemd unmount filesystems in order; avoid hard resets that leave journals inconsistent.
-
Back up.
fsckcan and does discard unrecoverable data intolost+found. Tested backups are your real safety net. -
Schedule periodic checks. Tune the mount-count and time intervals so ext4 fscks itself during a planned boot rather than after a crash:
tune2fs -c 30 /dev/sdb1 # force fsck every 30 mounts tune2fs -i 30d /dev/sdb1 # ...or at least every 30 days
Quick Command Reference
# Diagnose
mount | grep ' ro,'
findmnt -o TARGET,FSTYPE,OPTIONS /
dmesg -T | grep -i 'ext4\|remounting\|read-only'
tune2fs -l /dev/sdb1 | grep -i 'state\|error'
smartctl -H /dev/sdb # check the disk BEFORE repairing
# Repair a non-root filesystem
umount /data
fsck -y /dev/sdb1
mount /data
# Repair the root filesystem
touch /forcefsck && reboot # schedule fsck on next boot
# ...or from a rescue shell with / unmounted:
fsck -y /dev/sda1
mount -o remount,rw /
Conclusion
Read-only file system after a remount-ro is the kernel telling you it caught the filesystem doing something dangerous and pulled the emergency brake. The wrong move is to release the brake with mount -o remount,rw and keep driving. The right move is a disciplined sequence: read dmesg and tune2fs to confirm it was an errors-based remount, verify the underlying disk is healthy with smartctl, then run fsck on an unmounted filesystem (or schedule it for the next boot on root). Repair the corruption, confirm the state is clean, and only then bring the volume back read-write. If SMART says the disk is dying, image it first and recover from the copy — never fsck a failing drive in place.
Get 500 Battle-Tested DevOps AI Prompts — Free
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.