🚨 OpenStack Error Guide: 'Image stuck in saving' or 'killed' when uploading to Glance

Overview

When you upload an image to Glance and it never reaches active — instead hanging in saving or dropping straight to killed — the data path between the glance-api worker and its store backend has failed. The error usually looks like one of these:

500 Internal Server Error: Failed to upload image data due to internal error
HTTPInternalServerError (HTTP 500): Error in store configuration. Adding images to store is disabled.
Image <id> could not be found after upload. The image may have been deleted during the upload, cleaning up the chunks uploaded.

or, when the bytes were written but the checksum did not match:

glance.api.v2.image_data Backend storage for image <id> disconnected after writing only 0 bytes
BadStoreUri: Invalid checksum: expected <a>, got <b>

This happens when the store backend (file, Swift, RBD/Ceph, or Cinder store) is full or misconfigured, when the worker lacks write permission, when a worker crashes or the client disconnects mid-upload, when checksums mismatch, or when the project image quota is exceeded. An image in killed means Glance accepted the request, started writing, and then aborted.

Symptoms

• openstack image create --file ... hangs for the duration of the upload, then returns HTTP 500.
• The image is created in the registry but sits in saving forever and never becomes active.
• The image flips to killed immediately after the upload starts.
• Booting from a recently uploaded image fails because the image has zero size.

openstack image list

+--------------------------------------+--------------------+--------+
| ID                                   | Name               | Status |
+--------------------------------------+--------------------+--------+
| 1b2c3d4e-5f60-7a81-9b02-c3d4e5f60718 | ubuntu-22.04       | active |
| 8f7e6d5c-4b3a-2918-0f7e-6d5c4b3a2918 | rocky-9-golden     | killed |
| a0b1c2d3-e4f5-6071-8293-a4b5c6d7e8f9 | debian-12-template | saving |
+--------------------------------------+--------------------+--------+

openstack image show a0b1c2d3-e4f5-6071-8293-a4b5c6d7e8f9 -c status -c size -c checksum

+----------+--------+
| Field    | Value  |
+----------+--------+
| status   | saving |
| size     | None   |
| checksum | None   |
+----------+--------+

A size and checksum of None on an image stuck in saving means no data ever landed in the store.

Common Root Causes

1. Store backend full

The most common cause: the destination ran out of space mid-write.

# file store
df -h /var/lib/glance/images
# Ceph (RBD) store
ceph df --id glance
# Swift store
openstack object store account show

Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb1       100G  100G     0 100% /var/lib/glance/images

2026-06-23 15:02:11.881 7 ERROR glance.api.v2.image_data [req-...] Failed to upload image data due to HTTP error:
[Errno 28] No space left on device

2. Store misconfigured or disabled

A wrong store name, missing pool, or disabled store stops uploads before any bytes flow.

# Kolla-Ansible
docker exec glance_api grep -E 'enabled_backends|default_backend' /etc/glance/glance-api.conf
# Traditional
grep -E 'enabled_backends|default_backend' /etc/glance/glance-api.conf

2026-06-23 15:04:30.207 7 ERROR glance.api.v2.image_data Error in store configuration.
Adding images to store is disabled. Store for scheme rbd not found.

3. Permissions on the store

The worker process must be able to write the store. For the file store that means filesystem ownership; for Ceph it means a working keyring.

# file store ownership
ls -ld /var/lib/glance/images
# Ceph keyring usable by the glance client?
ceph auth get client.glance
rbd -p images ls --id glance

2026-06-23 15:06:44.512 7 ERROR glance.api.v2.image_data PermissionError:
[Errno 13] Permission denied: '/var/lib/glance/images/a0b1c2d3-e4f5-6071-8293-a4b5c6d7e8f9'

4. glance-api worker crash or restart

If a worker dies mid-upload, the image is left in saving and is later reaped to killed.

# Kolla-Ansible
docker logs glance_api 2>&1 | grep -iE "worker|killed|traceback" | tail
# Traditional
journalctl -u openstack-glance-api | grep -iE "worker|killed|traceback" | tail

2026-06-23 15:09:18.330 7 ERROR glance Worker exited prematurely: signal 9 (SIGKILL)
2026-06-23 15:09:18.901 7 INFO glance.api.v2.image_data Image a0b1c2d3-... is being set to 'killed'

A SIGKILL often means the OOM killer reaped the worker on a large upload.

5. Interrupted upload / client disconnect

A dropped client connection or proxy timeout aborts the transfer and Glance cleans up the partial chunks.

docker logs glance_api 2>&1 | grep -i "disconnected\|cleaning up" | tail

2026-06-23 15:11:55.044 7 INFO glance.api.v2.image_data Backend storage for image a0b1c2d3-...
disconnected after writing only 0 bytes
2026-06-23 15:11:55.046 7 INFO glance.api.v2.image_data Cleaning up 0 chunks for partial upload

Check for a proxy/load-balancer timeout that is shorter than the upload duration (e.g. HAProxy timeout client).

6. Checksum mismatch

If the bytes written do not match the expected checksum, Glance rejects the image.

docker logs glance_api 2>&1 | grep -i "checksum" | tail

2026-06-23 15:13:40.770 7 ERROR glance.api.v2.image_data BadStoreUri: Invalid checksum.
Image data checksum verification failed: expected 5d41402abc4b2a76b9719d911017c592, got d41d8cd98f00b204e9800998ecf8427e

This typically indicates a truncated file or corruption on the wire.

Bonus: Project image quota exceeded

Glance enforces per-project image count and storage quotas.

openstack registered limit list --service glance 2>/dev/null || \
  grep -E 'image_count_total|image_size_total' /etc/glance/glance-api.conf

413 Request Entity Too Large: The size of the data <bytes> will exceed the limit.
ImageSizeLimitExceeded: The provided image size must be less than the maximum image size.

Diagnostic Workflow

Step 1: Read the image’s registry record

openstack image show <image-id> -c status -c size -c checksum

size: None means nothing was written; a non-zero size with killed means the write started then aborted.

Step 2: Tail the glance-api logs and match the request ID

# Kolla-Ansible
docker logs glance_api 2>&1 | tail -n 80
# Traditional
journalctl -u openstack-glance-api -n 80 --no-pager

Grab the req-... ID from the upload and grep it to follow the single request.

Step 3: Check store capacity and configuration

df -h /var/lib/glance/images        # file
ceph df --id glance                 # rbd
docker exec glance_api grep -E 'enabled_backends|default_backend' /etc/glance/glance-api.conf

Step 4: Verify store permissions

ls -ld /var/lib/glance/images       # file ownership
rbd -p images ls --id glance        # ceph write/list access

Step 5: Check for worker OOM and proxy timeouts

dmesg -T | grep -i "killed process" | grep -i glance | tail
# HAProxy / load balancer in front of glance-api
grep -A3 'frontend glance_api' /etc/haproxy/haproxy.cfg

Example Root Cause Analysis

A user reports that uploading a 30 GB golden image always ends in killed, while small CirrOS images upload fine. The scenario:

1. openstack image show reports status: killed, size: 0 — the write aborted very early.

2. The glance-api log shows the worker dying:

2026-06-23 15:09:18.330 ERROR glance Worker exited prematurely: signal 9 (SIGKILL)
2026-06-23 15:09:18.901 INFO  glance.api.v2.image_data Image 8f7e6d5c-... is being set to 'killed'

3. Investigation — dmesg -T | grep -i "killed process" confirms the OOM killer reaped the glance-api worker. The deployment was using a store driver path that buffered the whole upload in memory on a host with only 4 GB free, so the 30 GB image blew past available RAM.

4. Fix — switch the upload to a streaming/chunked path and give the worker more headroom. For a file store, ensure glance_store writes in chunks; for large images, raise the container/host memory and confirm df -h /var/lib/glance/images has the room:

df -h /var/lib/glance/images          # confirm destination space
docker restart glance_api             # or systemctl restart openstack-glance-api
openstack image delete 8f7e6d5c-4b3a-2918-0f7e-6d5c4b3a2918   # remove the killed record

The re-upload now streams to disk and reaches active.

Prevention Best Practices

• Alert when any Glance store crosses 80% usage: df -h /var/lib/glance/images for file, ceph df pool usage for RBD, account usage for Swift.
• Monitor glance-api worker restarts and OOM kills from dmesg/container logs so a single bad upload does not silently keep killing images.
• Set load-balancer timeout client/timeout server longer than your largest expected image upload to avoid mid-transfer disconnects.
• Track per-project image count and size quota usage and warn before users hit ImageSizeLimitExceeded.
• Feed store-capacity and worker-health signals into a triage runbook or an automated workflow like /dashboard/incident-response/ so the first responder lands on the store backend, not the registry.

# Store health snapshot
df -h /var/lib/glance/images
ceph df --id glance
openstack image list --status killed -f value -c ID -c Name

Quick Command Reference

# Status and registry record
openstack image list
openstack image show <image-id> -c status -c size -c checksum

# Logs (Kolla-Ansible)
docker logs glance_api 2>&1 | tail -n 80
# Logs (traditional)
journalctl -u openstack-glance-api -n 80 --no-pager

# Store capacity and config
df -h /var/lib/glance/images
ceph df --id glance
docker exec glance_api grep -E 'enabled_backends|default_backend' /etc/glance/glance-api.conf

# Permissions
ls -ld /var/lib/glance/images
rbd -p images ls --id glance

# Worker / proxy
dmesg -T | grep -i "killed process" | grep -i glance | tail
grep -A3 'frontend glance_api' /etc/haproxy/haproxy.cfg

# Cleanup
openstack image delete <image-id>
docker restart glance_api   # or systemctl restart openstack-glance-api

Conclusion

When a Glance image is stuck in saving, flips to killed, or the upload fails, the typical root causes are:

1. The store backend (file, Swift, RBD, or Cinder store) is full.
2. The store is misconfigured or disabled in glance-api.conf.
3. The worker lacks write permission on the store (filesystem ownership or Ceph keyring).
4. A glance-api worker crashed mid-upload, often via the OOM killer.
5. The client connection or a proxy timeout interrupted the transfer.
6. A checksum mismatch (truncation/corruption) or an exceeded project image quota.

Read the registry record first, follow the request ID through the glance-api log, then verify store capacity, permissions, and worker health. More Glance and image troubleshooting lives under /categories/openstack/.