GitLab Environments & Deployments Debug Prompt

You are a senior DevOps engineer who has built production CD workflows with GitLab environments — staging/prod tiers, dynamic environments, review apps, protected envs with manual approval. You know that the `environment:` keyword in a job is what creates a deployment record, and that scope is what links variables to environments.

I will provide:
- The symptom (deployment not appearing, environment shows wrong URL, stop job not firing, protected env blocking, deployment stuck in `running`)
- The job's `environment:` block
- The environment from GitLab UI (Operate → Environments)
- Whether the environment is protected
- Deployment history if relevant

Your job:

1. **Verify how environments are created**:
   - A job with `environment: name` creates/updates the environment AND a deployment record
   - `environment: name + url` lets you click through to the deployed URL
   - `environment.action: start | stop | prepare | verify | access` controls what kind of event
   - **Dynamic environments**: `name: review/$CI_COMMIT_REF_SLUG` — one per branch
2. **Decode `action:`**:
   - `start` (default) — creates a deployment
   - `prepare` — accesses env (gets vars) but doesn't deploy (useful for "prepare" steps)
   - `verify` — verifies a deployment (e.g., smoke test)
   - `access` — accesses environment without deploy
   - `stop` — removes the environment (used for review apps cleanup)
3. **For dynamic envs / review apps**:
   - Create job: `environment: { name: review/$CI_COMMIT_REF_SLUG, url: ..., on_stop: stop-review }`
   - Stop job: separate job with `environment: { name: ..., action: stop }`, typically `when: manual` AND `stage: cleanup`
   - `auto_stop_in: 1 week` — auto-cleanup
   - When MR closes, the stop job is triggered
4. **For protected environments**:
   - Project Settings → CI/CD → Protected environments
   - Only specific users/groups can deploy
   - Approval rules: require N approvals before deploy job can run
   - Common gotcha: deploy job has `when: manual` but the user clicking isn't in the approver list → 403
5. **For deployment tiers** (production, staging, testing, development):
   - `deployment_tier:` keyword
   - Used in DORA metrics calculation
   - Helps GitLab UI categorize
6. **For deployment stuck in "running"**:
   - Pipeline job finished but deployment shows running → environment system not getting update
   - Usually means missing `environment:` block or unexpected stop/start ordering
   - For deployments with their own lifecycle (like ArgoCD), GitLab can't know they finished
7. **For environment URL not updating**:
   - `environment.url:` is set at job time; if dynamic, must be resolvable
   - For LB-routed reviews: usually a wildcard DNS pointing to ingress
8. **For environment scope on variables**:
   - Variables can be scoped to specific environments (`*` matches all)
   - Production-only variables must have `production` scope AND a job with `environment: production`
   - Without env scope or with wrong scope, vars are empty

Mark DESTRUCTIVE: deleting an environment (loses deployment history), changing tier (DORA metric impact), stopping protected environment without approver flow.

---

Symptom: [DESCRIBE]
Job's `environment:` block:
```yaml
[PASTE]
```
Protected env settings: [DESCRIBE]
Recent deployment history (from UI):
[DESCRIBE]
Variable scope (if relevant):
```
[PASTE — Project CI/CD → Variables, scope column]
```

Why this prompt works

Environments in GitLab are a separate object lifecycle from jobs. Many “deployment didn’t happen” debugging conversations are actually environment-config issues. This prompt walks the environment → job → deployment chain.

How to use it

1. Always include the environment: block from the job spec.
2. Check the environment in the UI — current status, URL, deployment count.
3. For protected envs, confirm approvers list.
4. For dynamic envs, check the slug expansion ($CI_COMMIT_REF_SLUG).

Useful commands

# List environments via API
curl --header "PRIVATE-TOKEN: <t>" \
    "https://gitlab.example.com/api/v4/projects/<id>/environments" | jq

# Get specific environment
curl --header "PRIVATE-TOKEN: <t>" \
    "https://gitlab.example.com/api/v4/projects/<id>/environments/<env-id>" | jq

# List deployments
curl --header "PRIVATE-TOKEN: <t>" \
    "https://gitlab.example.com/api/v4/projects/<id>/deployments" | jq

# Stop an environment via API
curl --request POST --header "PRIVATE-TOKEN: <t>" \
    "https://gitlab.example.com/api/v4/projects/<id>/environments/<env-id>/stop"

# Delete (after stopping)
curl --request DELETE --header "PRIVATE-TOKEN: <t>" \
    "https://gitlab.example.com/api/v4/projects/<id>/environments/<env-id>"

Patterns

Static environments (staging + production)

deploy-staging:
  stage: deploy
  script: ./deploy.sh staging
  environment:
    name: staging
    url: https://staging.example.com
    deployment_tier: staging
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy-production:
  stage: deploy
  script: ./deploy.sh production
  environment:
    name: production
    url: https://example.com
    deployment_tier: production
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+\.\d+\.\d+$/
      when: manual

Review apps (dynamic per branch)

review:
  stage: review
  script: ./deploy-review.sh
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.review.example.com
    on_stop: stop-review
    auto_stop_in: 1 week
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

stop-review:
  stage: cleanup
  variables:
    GIT_STRATEGY: none           # nothing to check out
  script: ./teardown-review.sh
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  when: manual                    # MR-close also triggers this
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: manual

Protected env with approval

deploy-production:
  stage: deploy
  script: ./deploy.sh prod
  environment:
    name: production
    url: https://app.example.com
    deployment_tier: production
  rules:
    - if: $CI_COMMIT_TAG
      when: manual
# In project Settings → CI/CD → Protected environments:
#   - production
#   - Allowed to deploy: maintainer + specific user list
#   - Required approvals: 2

Common findings this catches

• Deployment doesn’t appear in Environments UI → job missing environment: block.
• Stop job won’t fire on MR close → not configured with when: manual AND on_stop reference.
• environment.url shows old URL → environment URL set at deploy time; redeploy to update.
• Protected env approval rejecting apply → user clicking isn’t in approver list.
• Variables empty in deploy job → variable scoped to production but job’s environment.name doesn’t match.
• Environment shows “Available” but URL 404 → app removed manually; environment record stale; click “Stop.”
• Many dangling review/* envs → auto_stop_in not set.

When to escalate

• Cross-environment deploy collisions (staging and prod sharing infra) — design issue; separate at the infra level.
• Production deploys requiring more than GitLab’s approval workflow can model — consider ServiceNow / external approval integration.
• DORA metrics not matching reality — verify deployment_tier set on production deployments; coordinate with metrics team.