🚨 Docker Error Guide: 'error creating overlay mount ... no such file or directory' — overlay2 Corruption

Overview

Docker fails to start a container when it cannot assemble the overlay2 filesystem for the image’s layers. It appears on docker run or docker start:

Error response from daemon: error creating overlay mount to /var/lib/docker/overlay2/3f9a1c.../merged: no such file or directory

overlay2 stacks read-only image layers plus a writable layer into a single merged view using kernel lowerdir/upperdir references stored in each layer’s link and lower files. When one of those referenced directories is missing, the mount cannot be built and the kernel returns ENOENT. This signals missing or corrupted layer data under /var/lib/docker/overlay2, not a networking or registry problem.

Symptoms

• A container that ran fine yesterday now fails to start with error creating overlay mount ... no such file or directory.
• The failure names a specific overlay2/<id>/merged path.
• Multiple containers from the same image fail together.
• The error follows an unclean shutdown, a full disk event, or manual deletion under /var/lib/docker.
• docker start retries fail identically; re-pulling the image sometimes resolves it.

Common Root Causes

• Manual or accidental deletion of directories under /var/lib/docker/overlay2 (a common mistake when reclaiming disk).
• Unclean daemon shutdown or host crash leaving the overlay2 metadata (link/lower files) inconsistent.
• A prior “no space left on device” event that truncated layer extraction, leaving partial directories.
• Filesystem corruption on the volume backing /var/lib/docker.
• Storage driver mismatch after changing storage drivers without clearing the old data root.
• Backup/restore of /var/lib/docker that did not preserve the overlay2 structure exactly.

Diagnostic Workflow

First, read the daemon logs to capture the exact failing layer and any I/O errors:

sudo journalctl -u docker --since '15 minutes ago' | grep -i 'overlay\|no such file\|error'

Identify the image and its layers so you know which container to inspect:

docker inspect myapp:1.4.2 --format '{{.GraphDriver.Data.MergedDir}}'
docker image inspect myapp:1.4.2 --format '{{json .RootFS.Layers}}'

Check whether the referenced overlay2 directory actually exists on disk:

sudo ls -ld /var/lib/docker/overlay2/3f9a1c*/merged
sudo ls /var/lib/docker/overlay2/ | head

Rule out an underlying filesystem or disk problem, which often accompanies overlay corruption:

sudo dmesg | grep -i error
df -h /var/lib/docker
df -i /var/lib/docker

If the layer directory is genuinely missing, the image data is corrupt. Remove the affected image/container and re-pull to rebuild the layers cleanly:

docker rm -f <container>
docker image rm myapp:1.4.2
docker pull registry.example.com/myapp:1.4.2

Example Root Cause Analysis

After a host ran out of disk and was cleaned up by hand, a container fails to start: error creating overlay mount to /var/lib/docker/overlay2/3f9a1c.../merged: no such file or directory. The daemon log confirms overlay2 cannot find a lower directory:

sudo journalctl -u docker --since '10 minutes ago' | grep -i overlay
# error creating overlay mount ... lowerdir=.../l/ABCD...: no such file or directory

Checking the referenced link target shows it is gone — someone rm-ed directories under overlay2 to free space:

sudo ls /var/lib/docker/overlay2/3f9a1c*/merged
# ls: cannot access ... : No such file or directory

Because the image’s layer data was deleted, no mount can be assembled. The safe fix is to discard the broken image and re-pull it so Docker rebuilds every layer:

docker rm -f myapp_web
docker image rm -f myapp:1.4.2
docker pull registry.example.com/myapp:1.4.2
docker run -d --name myapp_web registry.example.com/myapp:1.4.2

If many images are affected and the data root is beyond repair, the last resort is stopping Docker, moving /var/lib/docker aside, and letting the daemon recreate a clean data root — after backing up any named volume data first.

Prevention Best Practices

• Never delete directories under /var/lib/docker/overlay2 by hand; reclaim space with docker system prune and docker image prune instead.
• Keep byte and inode headroom on /var/lib/docker so layer extraction never truncates mid-write.
• Ensure clean daemon shutdowns; avoid hard power-offs of Docker hosts.
• Monitor dmesg and disk SMART health so filesystem corruption is caught early.
• When changing storage drivers, start from a clean data root rather than reusing overlay2 data.
• Back up named volumes independently rather than snapshotting the raw /var/lib/docker tree.

Quick Command Reference

# Capture the exact failing layer from the daemon log
sudo journalctl -u docker --since '15 minutes ago' | grep -i overlay

# Confirm whether the referenced overlay2 dir exists
sudo ls -ld /var/lib/docker/overlay2/3f9a1c*/merged

# Rule out disk / filesystem problems
sudo dmesg | grep -i error
df -h /var/lib/docker && df -i /var/lib/docker

# Discard the broken image and re-pull clean layers
docker image rm -f myapp:1.4.2
docker pull registry.example.com/myapp:1.4.2

Conclusion

error creating overlay mount ... no such file or directory means overlay2 cannot assemble a container’s filesystem because a referenced layer directory under /var/lib/docker/overlay2 is missing — usually from manual deletion, an unclean shutdown, or a prior disk-full event. Read the daemon log to find the failing layer, confirm the directory is gone, rule out filesystem corruption with dmesg, and re-pull the affected image to rebuild its layers. Never hand-delete under overlay2 and keep disk headroom to prevent recurrence. For more storage and runtime fixes, see the Docker guides.