🚨 GCP Error: 'Operation failed because another operation was already in progress' — Cause, Fix, and Troubleshooting Guide

Overview

Cloud SQL performs administrative operations (backups, config changes, maintenance, replica creation) one at a time per instance. Starting a second operation while one is still running returns a 409:

ERROR: (gcloud.sql.instances.patch) HTTPError 409:
Operation failed because another operation was already in progress.
There is already an in progress operation on the instance.

This is a concurrency/serialization conflict, not a permissions or config error. The instance is busy; your request was rejected rather than queued.

Symptoms

• gcloud sql / API admin calls return HTTP 409 with another operation was already in progress.
• Terraform google_sql_database_instance applies intermittently fail on 409.
• Rapid back-to-back changes (flags, tier, backup, replica) fail after the first.
• The instance shows a long-running PENDING/RUNNING operation.

Common Root Causes

1. A prior admin operation hasn’t finished

A backup, patch, maintenance, or restart is still in progress; instance changes can’t overlap.

2. Automated + manual changes colliding

A scheduled backup or Terraform run overlaps with a manual gcloud change.

3. Parallel IaC / pipeline runs

Two CI jobs (or a retry firing while the original still runs) both try to mutate the same instance.

4. Self-service maintenance window

A Google-initiated maintenance operation is holding the instance.

How to Diagnose

All read-only.

# What operation is currently running/pending on the instance?
gcloud sql operations list --instance=prod-db \
  --project=acme-prod-platform \
  --filter="status!=DONE" \
  --format="table(name, operationType, status, startTime)"

# Instance state (BUSY/RUNNABLE/MAINTENANCE)
gcloud sql instances describe prod-db \
  --project=acme-prod-platform \
  --format="value(state, settings.maintenanceWindow)"

# Full recent operation history for context
gcloud sql operations list --instance=prod-db \
  --project=acme-prod-platform --limit=5 \
  --format="table(operationType, status, startTime, endTime)"

A row with status: RUNNING (or PENDING) is the operation you must wait on.

Fixes

Wait for the in-progress operation to finish, then retry. Block on it explicitly:

OP=$(gcloud sql operations list --instance=prod-db \
  --project=acme-prod-platform --filter="status!=DONE" \
  --format="value(name)" --limit=1)

gcloud sql operations wait "$OP" --project=acme-prod-platform
# then re-run your original change

Serialize changes in automation. Batch multiple flag/config changes into a single patch instead of several, and ensure only one pipeline mutates the instance at a time (locking/concurrency groups).

Add retry-with-backoff on 409 in scripts and Terraform, since the conflict is transient.

What to Watch Out For

• The operation is not queued — a 409 means your request was dropped; you must retry after the current one completes.
• Combine multiple setting changes into one patch call to avoid triggering several sequential operations.
• Prevent concurrent Terraform/CI runs against the same instance (state locking, CI concurrency groups) to stop self-collisions.
• Google-initiated maintenance can also hold the instance; check the maintenance window before assuming your own automation caused it.

Related

• GCP Error: ‘Error 409: … already exists’ (Conflict)
• GCP Error: ‘googleapi: Error 412: Precondition Failed’
• GCP Error: ‘The resource … is already being used by another resource’
• More in the GCP error guides.