GitLab CI/CD Sparse-Checkout Monorepo Performance Prompt

You are a senior platform engineer who cuts CI clone time in large monorepos. You know GitLab Runner's git handling (`GIT_STRATEGY`, `GIT_DEPTH`, `GIT_SUBMODULE_STRATEGY`), git sparse-checkout cones, partial clone filters, and — critically — how to narrow a checkout WITHOUT starving a build of files it quietly depends on.

I will provide:
- Repo shape (size, number of top-level projects, submodules, LFS?)
- The slow job(s) and what path(s) they actually build/test
- Current `.gitlab-ci.yml` git-related config
- Symptoms (clone dominates job time / huge checkout / history walk needed)
- Runner type and whether build dirs are reused

Your job:

1. **Measure the baseline**: identify how much of job time is clone vs. work (`CI_DEBUG_TRACE` / runner timing) so the optimization is justified.
2. **Map each job's true file dependencies**: the project it builds PLUS shared config, tooling, generated files, and any path it reads at runtime. This cone, not just the project dir, is what must be checked out.
3. **Apply the right narrowing per job**:
   - `GIT_DEPTH` per job — shallow where no history is needed, deeper where `git describe`/changed-file detection is used.
   - Sparse-checkout cone via `GIT_STRATEGY` + a `pre_clone_script`/manual sparse setup, or partial clone `--filter=blob:none` for blob-lazy fetch.
   - `GIT_SUBMODULE_STRATEGY` and `GIT_SUBMODULE_PATHS` to fetch only needed submodules.
4. **Guard history-dependent steps**: any job doing `git describe`, changelog, or `git diff` for path-scoped rules needs enough depth or an unshallow step.
5. **Handle changed-path detection**: if jobs use `rules:changes`, ensure the fetch depth supports the comparison base (MR pipelines need the target branch reachable).
6. **Keep it debuggable**: document each job's cone so an excluded-file failure is traceable; prefer fresh clones while tuning.
7. **Report the win**: expected clone-time reduction and the risk (what each job now assumes is present).

Deliverables:
- Per-job git config (`GIT_DEPTH`, sparse cone, submodule scope)
- The sparse-checkout setup block (runner variables or `pre_clone_script`)
- The list of files each narrowed job still depends on (the cone)
- History-walk safeguards for the jobs that need them

Mark DESTRUCTIVE or RISKY: cones that might exclude silently-read files, `GIT_DEPTH: 1` on jobs that walk history, and partial clone rollout without measurement.

---

Repo shape: [size, projects, submodules, LFS?]
Slow job(s) + build path: [DESCRIBE]
Current git CI config (YAML): [PASTE]
Symptoms: [clone-dominated / huge checkout / history needed]
Runner + reused dirs?: [DESCRIBE]

Why this prompt works

In a big monorepo, most jobs spend more time cloning than building — but naive narrowing breaks builds that quietly read shared files. This prompt insists on mapping each job’s real dependency cone (project + shared config + tooling) before narrowing, and separately guards the jobs that walk history, so you get the clone-time win without the “file not found” mystery.

How to use it

1. Measure clone-vs-work time to justify the change.
2. Map each job’s real file cone, not just its project dir.
3. Apply per-job depth, sparse cone, and submodule scope.
4. Keep enough history for jobs that use git describe/changed-path rules.

Useful commands

# Set up a sparse cone manually (in a job)
git sparse-checkout init --cone
git sparse-checkout set projects/api libs/shared tools/ci

# Partial clone — defer blob download until a file is touched
git clone --filter=blob:none --sparse "$CI_REPOSITORY_URL" repo

# Unshallow just enough for a history walk
git fetch --deepen=50
git describe --tags   # needs tags reachable

# See what sparse-checkout currently includes
git sparse-checkout list

GitLab CI patterns

Per-job shallow depth + submodule scope

build-api:
  variables:
    GIT_DEPTH: "1"                        # this job needs no history
    GIT_SUBMODULE_STRATEGY: none          # api build uses no submodules
  script:
    - cd projects/api && make build

release:
  variables:
    GIT_DEPTH: "0"                        # full history for git describe/changelog
  script:
    - git describe --tags
    - ./scripts/changelog.sh

Sparse cone via runner pre_clone_script (config.toml)

[[runners]]
  pre_clone_script = """
    git config --global core.sparseCheckout true
  """

build-web:
  variables:
    GIT_STRATEGY: clone                   # fresh clone while tuning cones
  before_script:
    - git sparse-checkout init --cone
    - git sparse-checkout set projects/web libs/ui tools/ci   # the real cone
  script:
    - cd projects/web && npm ci && npm run build

Changed-path detection needs the target branch reachable

lint-changed:
  variables:
    GIT_DEPTH: "50"                       # enough to reach the MR target base
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - "projects/web/**/*"
  script:
    - git fetch origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"
    - git diff --name-only "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"...HEAD

Common findings this catches

• A sparse cone that omits a shared config/generated file the build reads, causing distant “file not found” errors.
• GIT_DEPTH: 1 breaking git describe, changelog, or rules:changes base comparison.
• Every job full-cloning a multi-GB monorepo when it builds one project.
• Submodules fetched wholesale when a job needs none.
• Partial clone rolled out fleet-wide without measuring the lazy-fetch cost.

When to escalate

• Monorepo so large that even sparse clone is slow — consider scalar/partial-clone server settings or a repo-cache mirror with platform.
• Cross-project build dependencies that make cones fragile — may need a proper build graph (Bazel/Nx) rather than checkout tricks.
• Persistent history needs across many jobs — evaluate a shared warm cache volume on self-hosted runners.