🚨 Grafana Error Guide: 'database is locked' on SQLite — Fix Grafana DB Contention

Overview

Grafana defaults to an embedded SQLite database at /var/lib/grafana/grafana.db. SQLite allows only one writer at a time, so under concurrent writes — dashboard saves, alert state updates, session and annotation writes — a transaction that cannot acquire the write lock in time fails with “database is locked.”

The literal errors you will see:

database is locked

level=error msg="Failed to save dashboard" error="database is locked"

level=error logger=sqlstore error="database is locked (5) (SQLITE_BUSY)"

It is a concurrency/storage problem, not corruption. It shows up most under load, on slow or networked storage, or when more than one Grafana instance shares a single SQLite file.

Symptoms

• Intermittent “database is locked” on dashboard save, alert evaluation, or login.
• Errors spike under load or when alerting writes state frequently.
• Grafana runs on NFS/EFS or a slow volume.
• Multiple Grafana replicas point at the same grafana.db.

Common Root Causes

1. Concurrent writers on single-writer SQLite

Alerting state writes, annotations, and dashboard saves collide; one loses the write lock and errors.

2. busy_timeout too low or WAL disabled

Without a long enough busy_timeout (or with rollback-journal instead of WAL), a contended write fails instead of waiting.

3. Slow or networked storage

grafana.db on NFS/EFS or a throttled disk holds the lock far longer, widening the collision window; SQLite locking over NFS is unreliable.

4. Multiple Grafana instances on one SQLite file

Horizontal scaling (HA) with a shared SQLite file is unsupported — every replica is a competing writer.

5. A long-running transaction or backup holding the lock

A backup reading the file, or a slow migration, blocks writers.

Diagnostic Workflow

Step 1: Confirm SQLite is the backend and see the errors

grep -A3 '^\[database\]' /etc/grafana/grafana.ini
sudo journalctl -u grafana-server --no-pager | grep -iE "database is locked|SQLITE_BUSY|sqlstore" | tail -20
kubectl logs deploy/grafana -n monitoring | grep -iE "database is locked|SQLITE_BUSY" | tail -20
grep -i "database is locked" /var/log/grafana/grafana.log | tail -20

Step 2: Check storage type and replica count

df -T /var/lib/grafana                 # look for nfs/nfs4 fstype
mount | grep grafana
kubectl get deploy grafana -n monitoring -o jsonpath='{.spec.replicas}{"\n"}'

More than one replica on SQLite, or an nfs fstype, is a red flag on its own.

Step 3: Enable WAL and raise busy_timeout

# grafana.ini
[database]
type = sqlite3
path = grafana.db
wal = true
cache_mode = private

WAL lets readers proceed during a write and reduces lock contention. Grafana also supports a connection busy handling; ensure fast local storage backs it.

Step 4: Move the DB to fast local storage

# Stop Grafana, move grafana.db off NFS onto a local SSD path, update grafana.ini path, restart
sudo systemctl stop grafana-server
sudo mv /var/lib/grafana/grafana.db /local/ssd/grafana/grafana.db
# set [database] path accordingly, then:
sudo systemctl start grafana-server

Step 5: For HA or heavy load, migrate to Postgres/MySQL

# grafana.ini
[database]
type = postgres
host = pg.internal:5432
name = grafana
user = grafana
password = ${GF_DB_PASSWORD}
ssl_mode = require

Postgres/MySQL support concurrent writers and are required for multiple Grafana replicas.

Example Root Cause Analysis

A team scales Grafana to 3 replicas in Kubernetes for redundancy, all mounting the same PVC with grafana.db. Dashboard saves and logins start failing intermittently:

logger=sqlstore error="database is locked (5) (SQLITE_BUSY)"

Three replicas are three writers against a single SQLite file on a shared volume — exactly the unsupported HA-on-SQLite scenario. Whichever replica loses the write lock errors.

Fix: provision an external Postgres, point [database] type = postgres at it, and let all three replicas share that. SQLite’s single-writer limitation no longer applies and the lock errors vanish. Root cause: horizontally scaling Grafana on embedded SQLite — the correct fix is an external RDBMS, not tuning SQLite.

Prevention Best Practices

• Use SQLite only for single-instance, light-load Grafana; move to Postgres/MySQL for HA or heavy alerting/annotation write volume; see more Grafana guides.
• Never run multiple Grafana replicas against one SQLite file — it is unsupported and guarantees lock errors.
• Enable [database] wal = true and keep grafana.db on fast local storage, never NFS/EFS.
• Keep alerting/annotation write rates in mind; high-frequency writes are what push SQLite over the edge.
• Back up SQLite with sqlite3 .backup (which cooperates with WAL) rather than copying the file while Grafana writes.

Quick Command Reference

# Confirm backend and see the lock errors
grep -A3 '^\[database\]' /etc/grafana/grafana.ini
sudo journalctl -u grafana-server | grep -iE "database is locked|SQLITE_BUSY" | tail -20
kubectl logs deploy/grafana -n monitoring | grep -iE "database is locked" | tail -20

# Storage type and replica count (both common culprits)
df -T /var/lib/grafana ; mount | grep grafana
kubectl get deploy grafana -n monitoring -o jsonpath='{.spec.replicas}{"\n"}'

# Enable WAL (grafana.ini)
# [database]
# type = sqlite3
# wal = true

# Or migrate to Postgres (grafana.ini)
# [database] type = postgres / host / name / user / password

Conclusion

“database is locked” is SQLite telling you it cannot serialize the writes you are throwing at it — a concurrency and storage problem, not corruption:

1. Confirm SQLite is the backend and read the SQLITE_BUSY errors.
2. Check for the two big culprits: NFS/slow storage and multiple replicas on one file.
3. Enable WAL and move the DB to fast local storage for single-instance setups.
4. For HA or heavy write load, migrate to Postgres/MySQL — the only correct fix for concurrent writers.

Match the database backend to your deployment shape and this error disappears for good.