GitLab CI/CD `services:` & Docker-in-Docker Integration Testing Prompt
Stand up databases, brokers, and dependent containers as CI `services:` (or via DinD) for integration tests — health-gating, aliases, networking, and isolation per job.
- Target user
- Engineers running integration/E2E tests against real dependencies in CI
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior CI engineer who runs reliable integration tests against real Postgres, Redis, Kafka, and app containers in GitLab jobs without flaky "connection refused" races. I will provide: - The dependencies my tests need (DB, cache, queue, mock APIs, the app under test) - My test command and language - Runner executor (Docker, Kubernetes, shell) and whether privileged mode is allowed - Current flakiness or networking errors Your job: 1. **Choose `services:` vs Docker-in-Docker** — `services:` for "I need N sidecar containers reachable by alias" (simplest, no privileged needed); DinD/`docker:dind` only when tests must BUILD images or run `docker compose`/Testcontainers. Recommend one and justify. 2. **Wire `services:` correctly** — show the `services:` block with `name`, `alias`, and command/entrypoint overrides; explain how the job container reaches a service by its alias hostname (e.g. `postgres:5432`), and that `localhost` does NOT work on the Docker executor. 3. **Gate on readiness** — services start in parallel with the job, so add a wait loop (`pg_isready`/`redis-cli ping`/`nc -z`) in `before_script` instead of `sleep`; provide the snippet. This kills most flakiness. 4. **Configure DinD safely** — if needed: `docker:dind` service, `DOCKER_HOST=tcp://docker:2376`, TLS vars, privileged runner caveat, and prefer Testcontainers' `TC_HOST`/DinD networking or `DOCKER_TLS_CERTDIR`. Note the security cost of privileged. 4. **Isolate per job** — unique DB names/schemas per `CI_JOB_ID` so parallel jobs don't share state; ephemeral data, no cleanup needed since the container dies with the job. 5. **Pass credentials** — service env vars (`POSTGRES_PASSWORD`, etc.) via `variables:`, and how the test app reads matching connection vars. 6. **Debug** — how to see service logs (they're folded into the job log), diagnose image pull/auth issues, and cache base images. 7. **Validate** — run the job 10x and confirm zero connection-refused flakes; confirm tests actually hit the real service (not a mock). Output: (a) the job with `services:` (or DinD) configured, (b) the readiness wait `before_script`, (c) per-job isolation setup, (d) connection variables, (e) a flakiness/debug checklist. Bias toward: `services:` over DinD when possible, readiness gates over sleeps, and least-privilege (avoid privileged unless building images).