🧬 AI-Assisted Dynamic Child Pipelines for GitLab Monorepos

In a monorepo with twenty services, the worst thing your CI can do is build and test all twenty every time someone changes one. The fix in GitLab is the dynamic child pipeline: a parent job runs a script that detects what changed, generates a .gitlab-ci.yml for only the affected services, and triggers it as a child pipeline. It’s a genuinely elegant pattern — and writing the generator script is exactly the kind of fiddly code-and-templating work where AI saves real time. The model is a fast junior engineer that can draft the change-detection logic and the YAML-emitting code in minutes; you verify the generated YAML is valid and that the detection logic doesn’t silently skip a service that should have been tested. Here’s how I build these.

The parent/child structure

The parent pipeline has one job that generates YAML as an artifact, and a second that triggers it:

stages:
  - generate
  - trigger

generate_pipeline:
  stage: generate
  image: "python:3.12"
  script:
    - python ci/generate_child.py > child-pipeline.yml
  artifacts:
    paths:
      - "child-pipeline.yml"

run_child:
  stage: trigger
  trigger:
    include:
      - artifact: "child-pipeline.yml"
        job: "generate_pipeline"
    strategy: "depend"

strategy: depend makes the parent reflect the child’s status — if the child fails, the parent fails, which is what you want for MR gating. The model knows this structure well. The interesting part, and where AI earns its keep, is generate_child.py.

Have AI write the change-detection logic

The generator’s job is: figure out which services changed in this MR, then emit a child job for each. The change detection is the load-bearing part, and it’s easy to get subtly wrong. I prompt Claude: “Write a Python script that diffs against the target branch, maps each changed file to its owning service directory under services/, and outputs the set of affected services.” A reasonable result:

import subprocess, os

target = os.environ["CI_MERGE_REQUEST_TARGET_BRANCH_NAME"]
changed = subprocess.check_output(
    ["git", "diff", "--name-only", f"origin/{target}...HEAD"]
).decode().splitlines()

services = {
    path.split("/")[1]
    for path in changed
    if path.startswith("services/")
}

This is clean, but it has the classic monorepo trap baked in: shared dependencies. If services/payments imports a library in libs/money, a change to libs/money must trigger payments too — but this script only sees libs/... changed and triggers nothing. The model writes the naive version unless you specifically prompt it about shared-dependency propagation. This is the single most important thing to verify, because the failure mode is silent: a service that should have been tested simply isn’t, the pipeline goes green, and the bug ships. I always ask the model explicitly to “account for shared libraries that multiple services depend on” and then I verify the dependency map by hand.

Pro Tip: Build a “always run” fallback for high-risk changes. If the diff touches the root .gitlab-ci.yml, the generator script itself, a shared base Docker image, or a top-level dependency lockfile, generate the full pipeline, not a filtered one. Ask the AI to add this escape hatch. The cost of occasionally over-building on an infra change is trivial compared to the cost of silently skipping every affected service because the change was “outside any service directory.” When in doubt, build everything.

Generate the child YAML, then validate it

The second half of the generator emits the actual child YAML for each affected service. I have the model build it from a template:

template = """
test_{svc}:
  stage: test
  image: "python:3.12"
  script:
    - cd services/{svc} && make test
"""

print("stages:\n  - test")
for svc in sorted(services):
    print(template.format(svc=svc))

The danger with any string-templated YAML is producing invalid YAML — a service name with a hyphen breaking a job name, bad indentation, an unquoted string with a colon. The model’s templates usually work for the happy path and break on edge cases in service names. So the generator must validate its own output, and I have the AI add a step that parses the generated YAML with a real YAML library before it’s emitted, failing loudly if it’s malformed. A child pipeline that fails to parse gives a cryptic error in the parent, so catching it in the generator is far kinder. The hand-rolled-YAML risks are covered in depth in the dynamic child pipelines guide.

Handle the empty-pipeline case

A subtle one the model always forgets: what if nothing relevant changed? If an MR only touches a README, your services set is empty, and you’d generate a child pipeline with zero jobs — which GitLab rejects with an error, failing the pipeline for no good reason. The generator needs to emit a trivial no-op job (or have the parent skip the trigger) when there’s nothing to build:

if not services:
    print("stages:\n  - noop\nnoop:\n  stage: noop\n  script: ['echo nothing to build']")

I prompt the model for this explicitly because it never handles it unprompted — its happy-path bias means it assumes there’s always something to build. There isn’t, and the empty case fails pipelines on docs-only MRs until you fix it.

Keep secrets out of the generator and the chat

The generator script runs in CI and the child jobs it produces may need secrets — registry tokens, deploy keys. Those flow through GitLab’s CI/CD variables and are referenced by name ($REGISTRY_TOKEN) in the generated YAML; the values never get hardcoded into the generated file and never go into the AI chat while you’re building the generator. There’s an extra wrinkle here worth flagging: because the child YAML is generated, be careful the generator never interpolates a secret value into the output string. Reference variables by name in the template so GitLab expands them at child-runtime, rather than baking values into a generated artifact that gets stored and logged. The model can introduce this bug if you’re not watching; review the template for any place a real value could leak into output.

Verify before you trust the generator

A pipeline generator is infrastructure that decides what gets tested, so a bug in it means tests silently don’t run — the highest-stakes kind of CI bug. My verification before merging:

• Test the change-detection against several real diffs: a single-service change, a shared-lib change (must propagate!), a docs-only change (must no-op), and an infra change (must build-all).
• Confirm the generated YAML parses, by running the generator locally and linting its output.
• Run the whole parent/child flow on a branch and watch the child pipeline actually trigger and run the right jobs.
• Review the template for any path where a secret value could land in generated output.

This is firmly human-in-the-loop territory. The AI drafts the generator fast — the git diffing, the templating, the service mapping — but it ships the naive version that misses shared-dependency propagation and the empty-pipeline case every time unless prompted, and those are exactly the bugs that silently skip tests. You supply the dependency knowledge and the edge cases, and you verify the generator does the right thing across all four change scenarios before it gates a single MR. For a second pass on the generator code, point the code review dashboard at it.

Conclusion

Dynamic child pipelines are the right answer for monorepo CI, and AI is great at drafting the generator script that makes them work — the diffing, the service mapping, the YAML templating. But it consistently misses the things that matter most: shared-dependency propagation, the empty-pipeline edge case, and the risk of leaking secret values into generated output. Those are silent, test-skipping failures, so you must supply the dependency map, add the build-all fallback, validate the generated YAML, and verify all four change scenarios before merge. Fast junior engineer, human-in-the-loop, review before merge. More monorepo and pipeline guides are in the GitLab CI/CD category, with generator-script prompts in our prompts library.