🦭 Podman Error: 'fuse-overlayfs: not found' and 'failed to mount overlay' in Rootless Storage

Exact Error Message

$ podman pull docker.io/library/alpine:latest
Trying to pull docker.io/library/alpine:latest...
Error: writing blob: adding layer with blob "sha256:9fda8d8...":
processing tar file(exit status 1): potentially insufficient UIDs or GIDs
available in user namespace: Error during unshare: fuse-overlayfs: not found

A closely related variant appears when the helper exists but the mount itself is refused:

ERRO[0000] failed to mount overlay: operation not permitted
Error: kernel does not support overlay fs: 'overlay' is not supported over extfs
at "/home/deploy/.local/share/containers/storage/overlay": backing file system
is unsupported for this graph driver

What It Means

Overlay is a union filesystem: Podman stacks read-only image layers under a writable container layer so containers start instantly instead of copying a whole root filesystem. Mounting an overlay historically required real root, because the kernel would not trust an unprivileged user to control the layer stack. Rootless Podman therefore needs one of two things — a kernel new enough to permit overlayfs mounts inside a user namespace (the “native rootless overlay” path, which also requires the backing filesystem to support it), or the userspace fuse-overlayfs helper, which implements the same semantics through FUSE and needs no kernel privilege at all.

When neither is available, containers/storage cannot create the graph driver and the pull fails at the moment it tries to unpack the first layer. The error text varies with which stage failed: a missing binary produces fuse-overlayfs: not found, a missing /dev/fuse produces a permission error from the FUSE mount, and an unsupported backing filesystem (for example a home directory on NFS, or an ecryptfs/overlay-on-overlay nesting inside a CI container) produces backing file system is unsupported. All three land you in the same place: no usable storage driver for that user.

Common Causes

• The fuse-overlayfs package is simply not installed on the host.
• /dev/fuse is missing or not writable inside the container or CI runner where Podman is nested.
• ~/.config/containers/storage.conf sets driver = "overlay" but no mount_program, on a kernel without rootless native overlay.
• The user’s storage lives on a filesystem overlay cannot back — NFS, or another overlay mount in a nested-container CI setup.
• The user namespace is unusable: no subuid/subgid range, or newuidmap/newgidmap missing or lacking file capabilities.
• Storage was initialised with one driver and storage.conf was later changed to another, leaving an inconsistent graph root.

Diagnostic Commands

Ask Podman what driver and helper it is actually using — this is the single most useful check:

podman info --format '{{.Store.GraphDriverName}}'
podman info --format '{{.Store.GraphDriver}}'
podman info --format '{{.Store.GraphRoot}}'

Confirm the helper binary exists and is executable by the unprivileged user:

command -v fuse-overlayfs
fuse-overlayfs --version
ls -l /dev/fuse

Read the configuration files in precedence order — the per-user file wins over the system one for rootless:

cat ~/.config/containers/storage.conf 2>/dev/null
grep -vE '^\s*(#|$)' /etc/containers/storage.conf

Verify the user namespace prerequisites, since a broken subuid range produces overlay errors that look like driver problems:

grep "^$(id -un):" /etc/subuid /etc/subgid
podman unshare cat /proc/self/uid_map
ls -l /usr/bin/newuidmap /usr/bin/newgidmap
getcap /usr/bin/newuidmap /usr/bin/newgidmap

Check what filesystem the graph root sits on. nfs or a nested overlay here explains an otherwise mysterious failure:

stat -f -c %T ~/.local/share/containers/storage
findmnt -no FSTYPE -T ~/.local/share/containers/storage

Step-by-Step Resolution

1. Install the helper. On Fedora/RHEL and on Debian/Ubuntu respectively:

sudo dnf install -y fuse-overlayfs
sudo apt-get update && sudo apt-get install -y fuse-overlayfs

2. Point rootless storage at it explicitly in your per-user config. This overrides whatever the system file says:

# ~/.config/containers/storage.conf
[storage]
driver = "overlay"
runroot = "/run/user/1000/containers"
graphroot = "/home/deploy/.local/share/containers/storage"

[storage.options.overlay]
mount_program = "/usr/bin/fuse-overlayfs"

3. If Podman is itself running inside a container (CI runners, nested builds), expose the FUSE device and allow the mount:

podman run --rm --device /dev/fuse \
  --security-opt seccomp=unconfined \
  --security-opt label=disable \
  quay.io/podman/stable podman info --format '{{.Store.GraphDriverName}}'

4. Fix the user namespace if podman unshare fails. Allocate a subuid/subgid range and re-apply the migration:

sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 deploy
podman system migrate
podman unshare cat /proc/self/uid_map

5. Switching drivers requires clearing the existing graph root — Podman will not convert layers in place. This deletes all local images, containers, and volumes for that user:

podman system reset
podman info --format '{{.Store.GraphDriverName}}'

6. If overlay is genuinely unavailable (NFS home directory, restricted kernel), fall back to vfs deliberately rather than fighting it:

# ~/.config/containers/storage.conf
[storage]
driver = "vfs"

vfs performs a full copy of every layer instead of stacking them, so a 400 MB image with eight layers can consume several gigabytes and pulls become noticeably slower. Treat it as a last resort, and if you hit it, watch disk usage closely — see no space left on device. A better fix for NFS homes is to relocate graphroot to local disk.

Prevention

• Install fuse-overlayfs as part of your base image or host build alongside podman itself.
• Keep graphroot on local disk; never leave it on an NFS or network-mounted home directory.
• Bake --device /dev/fuse into CI job definitions that run Podman-in-Podman.
• Provision subuid/subgid ranges during user creation so rootless works on first login.
• Pin the driver and mount_program in a managed storage.conf rather than relying on autodetection.
• Run podman system reset intentionally during driver migrations instead of discovering the half-migrated state later.

Related Errors

• Error: cannot re-exec process — the user namespace could not be created at all, usually missing subuid ranges. See cannot re-exec process.
• Error: no space left on device — the graph root filled up, very common after a vfs fallback. See no space left on device.
• Error: OCI runtime error: crun: cannot open sd-bus — a cgroup/session problem rather than storage. See crun sd-bus.
• Error: volume is in use — storage cleanup blocked by a live reference, not a driver fault. See volume is in use.

Frequently Asked Questions

Do I still need fuse-overlayfs on a modern kernel? Not always. Newer kernels permit unprivileged overlayfs mounts in a user namespace, and Podman uses that native path when the backing filesystem supports it. Check podman info --format '{{.Store.GraphDriver}}' — if no mount_program is listed and pulls work, you are on the native path.

Why did it work as root but not as my user? Root mounts overlay directly through the kernel with full privilege and never needs the FUSE helper. Rootless has a different graph root, a different config file, and different mount requirements, so the two paths fail independently.

Is vfs safe to use in production? It is correct but expensive — every layer is a full copy, so disk usage and pull times multiply. It is acceptable for a small fixed image set on a constrained host, but for anything with frequent pulls, fix overlay instead.

Why did podman system reset wipe my images? Because the graph root’s layout is driver-specific and cannot be converted in place. Reset is the supported way to change drivers cleanly. For more container fixes, see the Podman guides.