GitLab CI/CD `include:` & Template Refactor Prompt

You are a senior DevOps engineer who has built and maintained pipeline-template libraries used by dozens of projects. You know which abstractions help and which produce tangled YAML that nobody can debug.

I will provide:
- A `.gitlab-ci.yml` (or a set of similar files across projects) the user wants to refactor
- The scope: single-project DRY-up, multi-project shared templates, or a centralized "CI library"
- Constraints: GitLab tier (Premium/Ultimate features available?), self-managed vs SaaS, who controls the template repo

Your job:

1. **Choose the right abstraction primitive** for each duplication:
   - **`include:`** = pull in YAML from another file. Use for cross-project sharing or to split a large file.
     - `include:local:` — within the same repo
     - `include:project:` — from another repo (file path + ref)
     - `include:remote:` — by URL (rare; use with care, no versioning)
     - `include:template:` — GitLab-bundled templates (e.g., `SAST.gitlab-ci.yml`)
     - `include:component:` — CI/CD Components (versioned, parameterized; Premium for private)
   - **`extends:`** = inherit from a hidden job (job name prefixed `.`). Use for "many jobs differ in one or two keys."
   - **`!reference [...]`** = literally insert a piece of YAML (a list element, a map). Use when you need fragments rather than whole-job inheritance.
   - **YAML anchors (`&` and `*`)** = old-school inheritance; less GitLab-aware than `extends:`/`!reference`. Avoid for new code.
2. **For multi-project sharing**, recommend the architecture:
   - **Template project** (private repo with versioned templates), used via `include:project:ref:` — best for org-wide DRY
   - **CI/CD Components** (GitLab Premium/Ultimate self-managed; Free on SaaS Beta) — preferred newer mechanism; supports inputs/outputs and a catalog
   - **External shared GitLab project** + tagged releases — caller projects pin to a tag (`ref: v1.4`), not `main`
3. **For include ordering and overrides**:
   - Includes are merged top-to-bottom; later definitions override earlier
   - The host project's own `.gitlab-ci.yml` content wins over included content
   - `extends:` is applied AFTER all merging — extending an included job works
   - Be explicit about override semantics for users of the template
4. **Refactor `extends:` correctly**:
   - Hidden jobs start with `.` (e.g., `.deploy:`)
   - Reusable hidden jobs commonly define `image`, `before_script`, `cache`, `rules` patterns
   - `extends: [.a, .b]` merges deeply; arrays REPLACE, maps merge
   - Avoid extending 3+ levels deep — debugging becomes painful
5. **`!reference` for fragments**:
   - Use when you want a script BLOCK shared but everything else different:
     ```yaml
     .install:
       script:
         - apt-get update
         - apt-get install -y curl
     job-a:
       script:
         - !reference [.install, script]
         - my-extra-command
     ```
6. **Versioning included templates**:
   - **NEVER `ref: main`** on a shared template — every push to that template's main is a breaking change for every caller
   - Pin to tag (`ref: v1.2.0`) or commit SHA
   - Consider CI/CD Components for explicit input contracts
7. **Anti-patterns to flag**:
   - Triple-nested `extends:` that nobody can mentally compose
   - `include:remote:` to an external URL (no versioning, security risk)
   - YAML anchors used to share between `extends:` (works but legacy)
   - Templates that depend on caller-defined variables without documentation
   - Hidden jobs (`.foo`) used directly in `needs:` (silently doesn't work)

Provide the refactor as concrete YAML diffs and explain the override contract.

---

Refactor scope: [single-project DRY / multi-project library / CI/CD Components]
GitLab tier + edition: [Free / Premium / Ultimate; SaaS / self-managed]
Current `.gitlab-ci.yml`(s):
```yaml
[PASTE — at least the repetitive parts]
```
Pain points: [DESCRIBE — what's hard to change, what's unclear to readers]

Why this prompt works

DRY-ing up CI YAML is a domain where there are 4-5 mechanisms (include:, extends:, !reference, anchors, CI/CD Components) and each has different merge semantics and best uses. Picking the wrong one produces an abstraction that’s worse than the duplication it replaced. This prompt forces a primitive-choice step before refactoring.

How to use it

1. Identify the specific duplication. Is it whole jobs that share most fields? Scripts that share preambles? Variables and rules used in many jobs?
2. Pick the primitive per duplication type. Whole jobs → extends:. Scripts → !reference. Cross-project → include:.
3. Decide the override contract. Document which parts are overridable and how.
4. Version external templates. Pin to tag, never to main.

Primitive selector

Duplication type	Use
5 jobs that differ only in environment name	extends: .deploy (hidden job)
Shared image:, before_script: across pipeline	default: block at top of file
Same install steps in 3 different scripts	!reference [.install, script]
Multiple projects share build/test pattern	include:project: (with versioned ref)
Org-wide CI library with inputs	CI/CD Component
Single project, file growing too big	include:local: to split
GitLab-bundled (SAST, DAST, License)	include:template:

Patterns

default: block (file-wide defaults)

default:
  image: node:20-alpine
  before_script:
    - npm ci --cache .npm --prefer-offline
  cache:
    key:
      files: [package-lock.json]
    paths: [.npm/, node_modules/]
  interruptible: true
  tags: [docker]

lint:   { script: npm run lint }
test:   { script: npm test }
build:  { script: npm run build }

extends: for job family

.deploy:
  image: alpine:3.20
  before_script:
    - apk add --no-cache curl
  script:
    - ./deploy.sh "$ENV"
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy-staging:
  extends: .deploy
  variables:
    ENV: staging
  environment: staging

deploy-prod:
  extends: .deploy
  variables:
    ENV: prod
  environment: prod
  rules:
    - if: $CI_COMMIT_TAG =~ /^v\d+/

!reference for script fragments

.aws-login:
  script:
    - aws --version
    - aws sts get-caller-identity

deploy:
  script:
    - !reference [.aws-login, script]
    - aws s3 sync ./dist s3://my-bucket/

invalidate-cdn:
  script:
    - !reference [.aws-login, script]
    - aws cloudfront create-invalidation --distribution-id $CDN_ID --paths '/*'

Multi-project include:

include:
  - project: 'platform/ci-templates'
    ref: v2.4.1                      # versioned!
    file:
      - '/templates/python.gitlab-ci.yml'
      - '/templates/security.gitlab-ci.yml'

In platform/ci-templates at tag v2.4.1:

# templates/python.gitlab-ci.yml
.python-base:
  image: python:3.12-slim
  before_script:
    - pip install --upgrade pip && pip install -r requirements-dev.txt
  cache:
    key: { files: [requirements-dev.txt] }
    paths: [~/.cache/pip/]

test:
  extends: .python-base
  script: pytest --junitxml=report.xml
  artifacts:
    reports: { junit: report.xml }

CI/CD Component (modern)

# Caller project .gitlab-ci.yml
include:
  - component: gitlab.example.com/platform/ci-components/python-test@v1
    inputs:
      python_version: "3.12"
      coverage_threshold: 80

# Component repo: templates/python-test/template.yml
spec:
  inputs:
    python_version:
      default: "3.12"
    coverage_threshold:
      default: 0
      type: number
---
test:
  image: python:$[[ inputs.python_version ]]-slim
  script:
    - pytest --cov --cov-fail-under=$[[ inputs.coverage_threshold ]]

Anti-patterns this catches

• include:remote: to a URL not under your control → security risk + zero versioning.
• ref: main on shared templates → silent breakage when template author commits.
• extends: 3+ levels deep (a extends b extends c extends d) → debugging requires a whiteboard.
• extends: on a hidden job referenced in needs: → silently doesn’t work; needs: ignores hidden jobs.
• YAML anchors AND extends: together → mixed merge semantics; pick one.
• Caller defines variables that template silently consumes → undocumented contract; rename the template’s “expected” variables explicitly.
• Template that’s so configurable everything’s a variable → readers can’t tell the actual behavior without running it.

Override contract example

For a template author, document this:

# templates/python.gitlab-ci.yml
# ---
# Required variables (from caller):
#   none
# Optional overrides (from caller):
#   PYTHON_VERSION (default "3.12")
#   PIP_INDEX_URL  (default https://pypi.org/simple)
#   COVERAGE_MIN   (default 0)
# Jobs you'll get:
#   test     — runs pytest with coverage
#   lint     — runs ruff + mypy
# To skip a job, override its `rules:` to `when: never`.

When to escalate

• A refactor that touches 10+ caller projects — coordinate; staged rollout.
• Adoption of CI/CD Components on self-managed — check tier requirements; verify catalog feature is enabled.
• Templates owned by a team that’s leaving the org — get ownership transferred before refactoring.