🚨 Docker Error Guide: 'Cannot kill container: did not receive an exit event' — Recover a Stuck Container

Overview

Docker reports this when it sends a signal to a container but never gets the confirmation that the container’s process actually exited, so it refuses to consider the kill complete:

Error response from daemon: Cannot kill container: 7f3c9a2b1e4d: tried to kill container, but did not receive an exit event

Docker delegates process lifecycle to containerd and its shim; the daemon waits for an exit event to travel back up that chain. When the event never arrives — because the process is wedged in the kernel or the daemon/containerd state is out of sync — docker kill, docker stop, and docker rm all hang or fail with this message.

Symptoms

• docker kill <id> and docker stop <id> fail with did not receive an exit event.
• docker rm -f <id> also fails; the container is stuck in a Removal In Progress or perpetual Up/Exited limbo.
• docker ps shows the container running, but the app is unresponsive.
• The container’s main process appears in D (uninterruptible sleep) or Z (zombie) state in ps.
• Restarting the container is impossible until the daemon or containerd is cycled.

Common Root Causes

• Process stuck in D-state — the container’s main process is blocked in an uninterruptible kernel wait (usually stuck I/O on NFS, a hung block device, or a FUSE mount) and cannot be signalled to death.
• dockerd/containerd desync — the daemon’s view of the container diverged from containerd’s after a crash, upgrade, or OOM, so the exit event routing is broken.
• Dead or orphaned shim — the containerd-shim that owns the container died or lost its connection, so no exit event is emitted.
• Zombie process — the process exited but was never reaped, leaving containerd waiting for an event that will not come.
• Kernel/storage hang — an overlay2 or volume backing store (e.g. a stalled network mount) freezes the process during teardown.
• containerd overload — a heavily loaded or partially hung containerd fails to deliver events promptly.

Diagnostic Workflow

First, find the container’s host PID and its process state — a D state means uninterruptible sleep (unkillable until the I/O unblocks):

docker inspect --format '{{.State.Pid}}' <id>
ps -o pid,stat,wchan,cmd -p $(docker inspect --format '{{.State.Pid}}' <id>)
# STAT column: D = uninterruptible sleep, Z = zombie, R/S = normal

If the process is in D-state, find what it is blocked on:

sudo cat /proc/$(docker inspect --format '{{.State.Pid}}' <id>)/stack 2>/dev/null
sudo cat /proc/$(docker inspect --format '{{.State.Pid}}' <id>)/wchan; echo

Check whether dockerd and containerd agree about the container:

sudo ctr -n moby containers ls | grep <id>
sudo ctr -n moby tasks ls | grep <id>

Read both daemons’ logs for the exit-event failure:

journalctl -u docker --since '15 min ago' | grep -i 'exit event\|kill'
journalctl -u containerd --since '15 min ago' | grep -i 'shim\|task\|<id>'

Confirm the shim is alive:

ps aux | grep -i 'containerd-shim' | grep <id>

Example Root Cause Analysis

An operator could not remove a data-processing container. Every docker stop and docker rm -f returned did not receive an exit event, and docker ps still showed it Up.

Inspecting the process state revealed the cause:

ps -o pid,stat,wchan,cmd -p $(docker inspect --format '{{.State.Pid}}' proc1)
#   PID STAT WCHAN            CMD
#  4821 D    nfs_wait_bit_...  /usr/bin/worker

The process was in D state blocked inside an NFS wait — the container had a bind-mounted NFS volume whose server had gone away. A D-state process cannot be killed by any signal, including SIGKILL, so Docker never received an exit event. The container could not be reaped until the underlying I/O resolved.

Recovery steps, in order of escalation:

# 1. Restore the stuck resource if possible (remount / recover the NFS server),
#    which lets the D-state process complete and exit on its own.
sudo mount -o remount /mnt/nfs

# 2. If dockerd/containerd are desynced, restart containerd, then docker.
sudo systemctl restart containerd
sudo systemctl restart docker

# 3. Verify the task/container are gone from containerd's view.
sudo ctr -n moby tasks ls | grep proc1 || echo 'task cleared'

After the NFS export was recovered the worker unblocked and exited cleanly; where the mount could not be restored, cycling containerd and then docker cleared the orphaned task and let the container be removed. A last-resort reboot is the only guaranteed cure for a truly wedged D-state process.

Prevention Best Practices

• Avoid bind-mounting fragile network storage (NFS/CIFS/FUSE) directly into containers without timeouts; a vanished server produces unkillable D-state processes.
• Use soft/timeo mount options on network filesystems so blocked I/O eventually errors instead of hanging forever.
• Keep Docker and containerd versions matched and upgrade them together to avoid state desync.
• Monitor for containers stuck in stopping/removing states and alert on them.
• Give containerd adequate resources; an overloaded containerd delays exit-event delivery.
• Prefer graceful application shutdown handling (trap SIGTERM) so processes exit before Docker has to force-kill them.

Quick Command Reference

# Host PID + process state (D = uninterruptible, unkillable)
ps -o pid,stat,wchan,cmd -p $(docker inspect --format '{{.State.Pid}}' <id>)

# What is a D-state process blocked on?
sudo cat /proc/<pid>/stack

# Do dockerd and containerd agree?
sudo ctr -n moby tasks ls | grep <id>

# Restart the runtime (containerd first, then docker)
sudo systemctl restart containerd && sudo systemctl restart docker

# Read the exit-event failure
journalctl -u containerd --since '15 min ago' | grep -i 'shim\|task'

# Last resort for a wedged D-state process
sudo reboot

Conclusion

Cannot kill container: did not receive an exit event means Docker signalled the container but never got confirmation it died — either because dockerd and containerd fell out of sync, or because the process is stuck in an uninterruptible D state (usually hung network/block I/O). Start with ps to read the process state: if it is D-state, fix or remove the underlying resource, since no signal can kill it. Otherwise restart containerd then docker to clear the orphaned task. A reboot is the guaranteed last resort.

For related lifecycle failures, see the Cannot kill container is not running guide, the container is restarting guide, and more fixes in the Docker guides.