🐳 Docker Error Guide: 'devmapper: Thin Pool has free data blocks less than minimum required'

Exact Error Message

$ docker run -d nginx
docker: Error response from daemon: devmapper: Thin Pool has 8092 free data blocks which is less than minimum required 8760 free data blocks. Create more free space in thin pool or use dm.min_free_space option to change behavior.

The same exhaustion shows up in the daemon logs and can also surface as:

devmapper: Thin Pool has 0 free metadata blocks which is less than minimum required
Error response from daemon: mkdir ... no space left on device

What It Means

The legacy devicemapper storage driver stores image and container layers inside a thin-provisioned LVM pool (or, in loop-lvm mode, inside sparse loopback files). Thin provisioning hands out virtual space lazily from a fixed backing pool. When the pool’s real data (or metadata) blocks run low, Docker refuses to create new containers to avoid corrupting the pool, and you get Thin Pool has <N> free data blocks which is less than minimum required.

dm.min_free_space (default 10%) is the safety threshold: Docker stops allocating once free space drops below it. So the error can fire while the pool still shows some free blocks. This is a capacity problem in the thin pool, not necessarily the host filesystem.

Common Causes

• The thin pool genuinely filled up from accumulated images, stopped containers, and dangling volumes.
• loop-lvm mode (the default fallback) uses sparse files that grew until the underlying filesystem ran out.
• The pool’s metadata device filled even though data blocks remained.
• Deleted containers left orphaned thin devices that were never reclaimed.
• The backing volume group has no free extents to auto-extend the pool.
• A single large image pull or build exceeded the remaining headroom in one shot.

Diagnostic Commands

Check the driver in use and the pool’s data/metadata usage:

docker info | grep -A20 'Storage Driver'

Look specifically at Data Space Used/Total and Metadata Space Used/Total in that output. Then inspect the LVM thin pool directly:

sudo lvs -o lv_name,data_percent,metadata_percent,lv_size

See how much space Docker itself is holding in images, containers, and volumes:

docker system df -v

Check whether the volume group has free extents to grow into:

sudo vgs

Step-by-Step Resolution

1. Reclaim space first; this often clears the error without touching LVM. Remove stopped containers, dangling images, unused networks, and build cache:

docker system prune -a

2. Remove unused volumes separately, since prune leaves named volumes alone by default (verify you do not need their data):

docker volume prune

3. Re-check pool usage. If data blocks are back above the threshold, retry the container:

sudo lvs -o lv_name,data_percent,metadata_percent
docker run -d nginx

4. If the pool is still tight and the volume group has free extents, extend the thin pool’s data and metadata:

sudo lvextend -L +20G /dev/docker-vg/docker-pool
sudo lvextend --poolmetadatasize +1G /dev/docker-vg/docker-pool

5. If the volume group itself is full, add a disk and grow the VG before extending the pool:

sudo pvcreate /dev/xvdf
sudo vgextend docker-vg /dev/xvdf
sudo lvextend -L +50G /dev/docker-vg/docker-pool

6. Confirm you are not on loop-lvm. If docker info shows Data file: /dev/loop0, you are in the slow, unreliable loopback mode. The right fix is to migrate off devicemapper entirely to overlay2:

{
  "storage-driver": "overlay2"
}

Put that in /etc/docker/daemon.json, then (this discards existing images/containers under devicemapper, so back up first):

sudo systemctl stop docker
sudo rm -rf /var/lib/docker/devicemapper
sudo systemctl start docker
docker info | grep 'Storage Driver'

Prevention

• Migrate to overlay2. Devicemapper is deprecated and overlay2 is the modern default that uses the underlying filesystem directly, avoiding thin-pool exhaustion altogether. The DevOps AI prompt library can generate a safe overlay2 migration checklist for a host.
• If you must stay on devicemapper, use direct-lvm (never loop-lvm) with a dedicated block device and monitor data_percent/metadata_percent.
• Schedule regular docker system prune and docker volume prune so layers and volumes do not silently accumulate.
• Alert on thin-pool utilization crossing ~70% so you can extend before hitting the dm.min_free_space wall.
• Keep spare extents in the volume group (or configure pool autoextend via lvm.conf) so the pool can grow automatically under pressure.

Related Errors

• no space left on device — the host filesystem or overlay2 backing store is full, a related but distinct capacity error.
• devmapper: Unknown option dm.min_free_space — a misconfigured storage-opt in daemon.json.
• Error response from daemon: failed to mount ... device or resource busy — a stuck devicemapper thin device, often after an unclean shutdown.
• there are no more loopback devices available — loop-lvm exhausted the host’s loop devices.

Frequently Asked Questions

Why does the error fire when the pool still shows free blocks? Docker enforces dm.min_free_space (default 10%) as a safety margin and stops allocating before the pool is truly empty, to avoid corrupting the thin pool. The free blocks you see are the reserved cushion.

Is devicemapper still recommended? No. It is deprecated in favor of overlay2, which most distributions default to. If you are hitting thin-pool errors, migrating to overlay2 is the durable fix rather than repeatedly extending the pool.

What is the difference between loop-lvm and direct-lvm? loop-lvm stores the pool in sparse files over loopback devices and is only meant for testing; it is slow and prone to exhaustion. direct-lvm uses a real block device and is the only supported production devicemapper mode.

Will docker system prune delete my data? It removes stopped containers, unused images, and build cache, and with -a all images not tied to a running container. It does not remove named volumes unless you also run docker volume prune, so back up volumes first.

How do I extend the pool without downtime? lvextend on the data and metadata LVs grows the pool online, provided the volume group has free extents or you add a new physical volume first. For more, see the Docker guides.