Terraform State Locking Debug Prompt

You are a senior platform engineer who has debugged stuck Terraform locks in CI/CD pipelines — DynamoDB, GCS, Azure Blob lease, GitLab managed state.

I will provide:
- Backend type
- Error message
- Recent CI activity

Your job:

1. **Lock semantics by backend**:
   - **S3 + DynamoDB**: lock row in DynamoDB table with LockID; auto-released on completion
   - **GCS**: object metadata; auto-released
   - **Azure Blob**: 60s lease, auto-renewed; released on close
   - **GitLab**: locks via API; releases on plan/apply complete
   - **Terraform Cloud**: workspace lock
2. **For "lock already held" error**:
   - Another run in progress — wait
   - Previous run crashed without releasing
   - Lock TTL not yet expired
3. **For force-unlock**:
   - `terraform force-unlock <LOCK_ID>` — for STUCK locks only
   - LOCK_ID from error message
   - Verify no concurrent run before forcing
4. **For CI/CD contention**:
   - Multiple pipelines targeting same state
   - Use `resource_group` (GitLab) to serialize
   - Use Terraform Cloud workspace
5. **For Atlantis / Spacelift**:
   - Locks managed at orchestration layer
   - May not align with Terraform's view
6. **For DynamoDB specifics**:
   - Lock row visible in table
   - Manual delete (last resort)
   - TTL not auto-set; orphans accumulate
7. **For lock during long-running apply**:
   - Slow resources can hold lock > 1h
   - Tune CI timeout
8. **For audit**:
   - Who held lock when
   - DynamoDB stream can capture
   - Terraform Cloud has UI

Mark DESTRUCTIVE: force-unlock during active apply (state corruption), manual DynamoDB row delete during apply, bypassing locking entirely (concurrent corruption).

---

Backend: [DESCRIBE]
Error: [PASTE]
Recent CI: [DESCRIBE]

Why this prompt works

Lock issues stop pipelines. This prompt walks resolution.

How to use it

1. Confirm no active run before force.
2. Use resource_group in GitLab.
3. For orphan locks, force-unlock.
4. Audit lock history.

Useful commands

# Show lock error (typical)
# Error: Error acquiring the state lock
# Lock Info:
#   ID:        4d6a8b2c-...
#   Path:      myorg-tfstate/env/prod/...
#   Operation: OperationTypeApply
#   Who:       gitlab-runner@runner-host
#   Created:   2026-05-30 14:00:00

# Force-unlock (only if confirmed orphan)
terraform force-unlock 4d6a8b2c-...

# Inspect DynamoDB lock (AWS)
aws dynamodb scan --table-name terraform-state-locks
aws dynamodb get-item --table-name terraform-state-locks \
    --key '{"LockID": {"S": "myorg-tfstate/env/prod/terraform.tfstate-md5"}}'

# Manual delete (LAST RESORT)
aws dynamodb delete-item --table-name terraform-state-locks \
    --key '{"LockID": {"S": "myorg-tfstate/env/prod/terraform.tfstate-md5"}}'

# GCS object inspect
gsutil ls -l gs://myorg-tfstate/env/prod/default.tflock
gsutil rm gs://myorg-tfstate/env/prod/default.tflock     # LAST RESORT

# Azure lease break
az storage blob lease break --container-name tfstate --blob-name prod.tfstate

CI/CD patterns

GitLab resource_group (serialize)

terraform-apply-prod:
  stage: apply
  resource_group: terraform-prod         # serializes across pipelines
  script:
    - terraform apply -auto-approve
  when: manual

Avoid concurrent runs (job-token allowlist)

# Settings → CI/CD → Pipelines for the same project
# Restrict via branch protection AND resource_group

Terraform Cloud workspace lock

# Via API
curl -X POST -H "Authorization: Bearer $TFC_TOKEN" \
    https://app.terraform.io/api/v2/workspaces/ws-xyz/actions/unlock

Common findings this catches

• “Lock acquisition timeout” → previous run hanging or orphan.
• Force-unlock + immediate concurrent run → corruption.
• DynamoDB table not exists → backend misconfigured.
• Lock from week ago → CI runner crashed; force.
• Multiple pipelines starting same time → resource_group.
• Apply taking 2h → split state; or accept long lock.
• Lock holder identity unclear → audit table.

When to escalate

• Production state corruption from race → restore from backup.
• Persistent CI orchestration issues — engineering.
• Multi-team state contention — split state design.