🚨 GitLab CI Error Guide: 'circular dependency detected' in needs — Fix the DAG

Overview

GitLab’s needs: keyword builds a directed acyclic graph (DAG) so jobs start as soon as their dependencies finish, instead of waiting for whole stages. “Acyclic” is the operative word: if job A needs B and B needs A (directly or through a chain), there is no valid start order and GitLab rejects the whole pipeline at compile time with a validation error:

Unable to create pipeline: circular dependency detected in the pipeline

You may also see these near-identical variants depending on GitLab version and the shape of the cycle:

jobs:test needs part of needs may not create a cycle
build job: needs config contains a circular dependency
'deploy' job needs 'test' job, but 'test' is not defined in prior stages

This is a YAML/DAG modeling error, not a runtime failure. The pipeline never starts; you fix it by editing the needs: graph.

Symptoms

• The pipeline fails immediately on push/MR with circular dependency detected and no jobs run at all.
• CI Lint (CI/CD → Editor → Validate) reports the config invalid with a circular-dependency message.
• The error appears right after adding or reordering a needs: entry, or after refactoring stages.
• A job needs: a job that runs in a later stage (or the same stage) creating an implicit cycle.
• A job accidentally lists itself in its own needs: array.

Common Root Causes

• Direct two-job cycle — A: needs: [B] and B: needs: [A].
• Indirect (transitive) cycle — A → B → C → A through several jobs; the loop is only visible when you trace the whole chain.
• Self-reference — a job lists its own name in needs: (often a copy-paste when duplicating a job).
• Cross-stage needs against stage order — needs: lets a job depend on any job, but if the depended-on job is in a later stage and it needs something from the earlier job, the combination of stage ordering + needs forms a cycle.
• Templated/extends merge — a shared template injects a needs: that, once merged into a concrete job, closes a loop the author never saw in one file.
• Matrix/parallel jobs — generated parallel:matrix: job names referenced in needs: in a way that loops back on the generator.

Diagnostic Workflow

1. Validate locally / with CI Lint first. Do not push to test — CI Lint catches the cycle without burning a pipeline:

# API CI Lint: reports the circular dependency without running anything
curl --silent --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$CI_API_V4_URL/projects/$CI_PROJECT_ID/ci/lint" \
  --data-urlencode "content@.gitlab-ci.yml" | jq '{valid, errors}'

2. Extract every needs: edge and draw the graph. List each job and what it needs, then look for a path that returns to its start. A quick way to dump the edges from your config:

# Print "job -> dependency" edges to eyeball the DAG
python3 - <<'PY'
import yaml
cfg = yaml.safe_load(open(".gitlab-ci.yml"))
for job, body in cfg.items():
    if isinstance(body, dict) and "needs" in body:
        for n in body["needs"]:
            dep = n["job"] if isinstance(n, dict) else n
            print(f"{job} -> {dep}")
PY

Any line where following the arrows comes back to the starting job is your cycle.

3. Check for self-references. Grep for jobs that need themselves:

grep -n "needs:" -A5 .gitlab-ci.yml

Confirm no job name appears inside its own needs: list.

4. Reconcile needs: with stage: ordering. A DAG job can only need jobs that will have started by the time it runs. Make sure the depended-on jobs are in the same or an earlier stage, and that no earlier-stage job needs a later one back. A clean, acyclic example:

stages: [build, test, deploy]

build:
  stage: build
  script: ["make build"]

unit-test:
  stage: test
  needs: ["build"]          # depends only on an earlier-stage job
  script: ["make test"]

integration-test:
  stage: test
  needs: ["build"]          # sibling in same stage, no loop
  script: ["make integration"]

deploy:
  stage: deploy
  needs: ["unit-test", "integration-test"]   # fan-in, still acyclic
  script: ["./deploy.sh"]

5. Inspect merged config from templates. If extends/include is in play, view the fully merged YAML so injected needs: are visible:

curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$CI_API_V4_URL/projects/$CI_PROJECT_ID/ci/lint" \
  --data-urlencode "content@.gitlab-ci.yml" \
  --data "include_jobs=true" | jq -r '.merged_yaml'

Trace needs: in the merged output — the cycle is often introduced by a shared template, not the local job.

Example Root Cause Analysis

A team split their monolithic test stage into DAG jobs to speed up feedback. After the change, every push failed instantly with circular dependency detected in the pipeline and zero jobs ran.

Diagnosis:

1. CI Lint confirmed the config was invalid with the circular-dependency error.
2. Dumping the edges produced: lint -> build, unit -> build, package -> unit, and — the culprit — build -> package. Someone had added needs: [package] to build so the build job could reuse a packaged fixture.
3. Tracing the arrows: build → package → unit → build. A three-job cycle.

Fix: the fixture the build job wanted actually belonged to a separate prepare-fixtures job. They removed needs: [package] from build and pointed build at prepare-fixtures instead, which sits in an earlier stage:

prepare-fixtures:
  stage: .pre
  script: ["./make-fixtures.sh"]
  artifacts: { paths: ["fixtures/"] }

build:
  stage: build
  needs: ["prepare-fixtures"]   # earlier stage, breaks the loop
  script: ["make build"]

CI Lint returned valid: true and the pipeline scheduled correctly. The cycle was gone because no edge pointed backward through the graph anymore.

Prevention Best Practices

• Validate with CI Lint (or the API) before pushing whenever you touch needs: — it catches cycles for free.
• Keep the mental model: a needs: arrow may only point at jobs in the same or earlier logical position; never let two jobs need each other.
• Give shared work (fixtures, build artifacts) its own upstream job in an early stage (.pre) rather than making peers depend on each other.
• When using extends/templates, review the merged YAML so injected needs: don’t silently close a loop.
• Avoid copy-pasting a job without renaming its needs: references — self-references are a top cause.
• Keep DAGs shallow and readable; a diagram or the edge-dump script above makes cycles obvious before they reach CI.

Quick Command Reference

# Validate config and surface the circular-dependency error
curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$CI_API_V4_URL/projects/$CI_PROJECT_ID/ci/lint" \
  --data-urlencode "content@.gitlab-ci.yml" | jq '{valid, errors}'

# View the fully merged YAML (templates/extends expanded)
curl -s --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$CI_API_V4_URL/projects/$CI_PROJECT_ID/ci/lint" \
  --data-urlencode "content@.gitlab-ci.yml" \
  --data "include_jobs=true" | jq -r '.merged_yaml'

# Dump job -> needs edges to spot the cycle (see the script in the workflow above)
python3 dump-needs-edges.py

# Find self-references and needs blocks quickly
grep -n "needs:" -A5 .gitlab-ci.yml

Conclusion

circular dependency detected in the pipeline is a compile-time DAG error: two jobs need each other directly or through a chain, so GitLab cannot pick a start order. It is not a flaky runtime issue — the same config will always fail until you break the loop. Dump the needs: edges, find the arrow that points backward, and move the shared dependency into an earlier standalone job. Validating with CI Lint every time you edit needs: keeps cycles out of your pipeline entirely.