Skip to content
DevOps AI ToolKit
Newsletter
All guides
Docker with AI By James Joyner IV · · 9 min read

Docker Error Guide: 'layers from manifest don't match image configuration' Pull Validation Failures

Fix Docker 'layers from manifest don't match image configuration': clear corrupted pull caches, registry mirror mismatches, and broken multi-arch manifests.

  • #docker
  • #troubleshooting
  • #errors
  • #registry

Exact Error Message

layers from manifest don't match image configuration is raised during docker pull, after the manifest and config blob have been fetched but before the image is committed:

docker pull registry.example.com/app:1.4.2

1.4.2: Pulling from app
Error response from daemon: layers from manifest don't match image
configuration

You may also see it surfaced from BuildKit or containerd-backed builds:

failed to copy: failed commit on ref "manifest-sha256:...":
layers from manifest don't match image configuration

The pull stops cold. Unlike a network error, retrying immediately usually reproduces it identically, because the mismatch is in the metadata Docker received, not in the connection.

What the Error Means

A Docker image has two coupled documents: the manifest (an ordered list of layer digests) and the image config (which records, among other things, a rootfs.diff_ids array describing the same layers in uncompressed form). When Docker assembles an image, it verifies that the layers referenced by the manifest correspond exactly to the diff_ids in the config. If the count differs, or the order/digests don’t line up, the image would be internally inconsistent — so the daemon refuses it with this error.

In practice the manifest and config disagree because one of them is stale, corrupted, or came from a different build than the other. Common culprits are a misbehaving registry mirror or proxy cache serving an old manifest with a new config (or vice versa), a partially-pushed image, or a corrupted local content cache. The image on the origin registry may be perfectly valid; the mismatch is in what reached your daemon.

Common Causes

  • Stale registry mirror/proxy cache. A pull-through cache returns a cached manifest that no longer matches the current config blob.
  • Partially pushed or overwritten tag. A push was interrupted, or a tag was re-pushed so the manifest now points at layers that don’t agree with the stored config.
  • Corrupted local content store. Bytes in /var/lib/docker (or containerd’s content store) were damaged, so the locally reconstructed manifest/config disagree.
  • Manifest list confusion. A multi-arch index where the per-arch manifest and config were built/pushed inconsistently.
  • Buggy intermediate proxy. A corporate proxy or registry that rewrites or truncates blobs in transit.
  • Mixed tooling push. An image assembled by one tool and re-tagged/pushed by another in a way that decoupled config and layers.

How to Reproduce the Error

The clean way to trigger it is a stale mirror: configure a pull-through cache, pull a tag, re-push a different image to the same tag at the origin, then pull again through the still-cached mirror:

# daemon.json points registry-mirrors at a caching proxy
docker pull mirror.example.com/app:latest   # caches old manifest
# (origin tag :latest is overwritten with a new build)
docker pull mirror.example.com/app:latest   # mirror serves old manifest + new config path
Error response from daemon: layers from manifest don't match image
configuration

Diagnostic Commands

Confirm whether a mirror or proxy is in play — this is the most common cause:

docker info | grep -iA3 'Registry Mirrors\|Insecure Registries'

Inspect the remote manifest directly (read-only) and note whether it’s an index or a single manifest:

docker manifest inspect registry.example.com/app:1.4.2 | head -40

Read the daemon log for the digests it was comparing:

journalctl -u docker --since "10 min ago" | grep -i 'manifest\|config\|diff_id\|mismatch'

If you use containerd image store, list what’s actually in the content store for the ref:

docker image inspect registry.example.com/app:1.4.2 2>/dev/null | grep -i 'digest\|id'

Step-by-Step Resolution

Cause: stale mirror/proxy cache. Pull directly from the origin registry, bypassing the mirror, to confirm the origin is healthy:

docker pull registry.example.com/app:1.4.2

If the direct pull succeeds, the mirror is at fault. Flush the offending tag from the cache (operation depends on your registry product) or temporarily remove the mirror from daemon.json and systemctl restart docker.

Cause: corrupted local content. Remove any partial local copy and re-pull cleanly:

docker rmi registry.example.com/app:1.4.2 2>/dev/null
docker pull registry.example.com/app:1.4.2

If your content store itself is damaged, that points to disk problems — check the host and the no space left on device guide for storage health.

Cause: bad push at origin. If even a direct pull fails, the image on the registry is inconsistent. The owner must rebuild and re-push it; you cannot repair it from the client side. For multi-arch, re-create the manifest list so each platform’s manifest and config agree.

Cause: rewriting proxy. If a corporate proxy mangles blobs, pull over a path that bypasses it or have the proxy pass registry traffic through untouched.

A worked example. A team’s nightly deploy starts failing on app:latest with this error, while a developer on a laptop pulls the same tag fine. The difference: the deploy host uses a pull-through mirror. The mirror had cached last week’s manifest, but the config blob was re-fetched fresh after a re-push, so the two no longer agreed. Removing registry-mirrors from the host’s daemon.json, restarting Docker, and re-pulling succeeded immediately. The lasting fix was configuring the mirror to expire manifests aggressively for mutable tags.

Prevention and Best Practices

  • Prefer immutable, digest-pinned references (app@sha256:...) for deploys so a re-pushed mutable tag can’t desync your cache.
  • Configure pull-through mirrors to expire or never cache mutable tags like latest.
  • Push images atomically; ensure CI fails the pipeline if a push is interrupted, and re-push fully.
  • Build and assemble multi-arch manifest lists with one toolchain so per-arch config and layers stay coupled.
  • Monitor disk health on hosts; corrupted content stores often trace back to failing storage.

Frequently Asked Questions

Does retrying the pull help? Rarely. The mismatch is in metadata, so an immediate retry reproduces it. It only “fixes itself” if a mirror’s cache later expires.

Is my image on the registry broken? Not necessarily. If a direct (non-mirrored) pull works, the origin is fine and a cache is to blame. If the direct pull also fails, the pushed image is inconsistent.

Can I force Docker to ignore the mismatch? No, and you shouldn’t want to — committing mismatched layers would produce an image that behaves unpredictably at runtime.

Why does it only happen on some hosts? Those hosts use a mirror or proxy that the working hosts don’t. Compare docker info registry settings across the fleet.

Free download · 368-page PDF

Download the Free 500-Prompt DevOps AI Toolkit

500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.

  • 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
  • Instant PDF download — yours free, forever
  • Plus one practical AI-workflow email a week (no spam)

Single opt-in · unsubscribe anytime · no spam.