GitLab CI/CD: Pipeline Anatomy, Docker-in-Docker & Recipes

GitLab CI/CD is one of the most complete CI platforms available today — the pipeline definition lives in the same repository as the code, the built-in container registry removes the need for a separate Docker Hub account, and the services keyword makes ephemeral sidecar containers (like a Docker daemon or a test database) trivially easy. These notes cover the patterns that come up on almost every real project.

`.gitlab-ci.yml` Anatomy

Every pipeline is defined in a .gitlab-ci.yml at the root of the repository. The top-level keys you’ll use on every project:

# .gitlab-ci.yml — minimal working example
stages:          # ordered list of stage names
  - build
  - test
  - deploy

variables:       # pipeline-wide environment variables
  IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

default:         # defaults applied to all jobs unless overridden
  image: alpine:3.19
  tags:
    - docker     # route jobs to runners with the "docker" tag

# A job belongs to a stage and runs its `script` steps sequentially
build:app:
  stage: build
  script:
    - echo "Building $IMAGE_NAME"

test:unit:
  stage: test
  script:
    - echo "Running unit tests"
  allow_failure: false   # default — fail the pipeline if this job fails

deploy:staging:
  stage: deploy
  script:
    - echo "Deploying to staging"
  environment:
    name: staging
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Stage order matters — jobs in stage n+1 only start after all jobs in stage n have passed. Jobs within the same stage run in parallel by default.

Key Built-in Variables

GitLab injects these into every job automatically — no configuration required:

Variable	Value	Typical Use
`$CI_REGISTRY`	`registry.gitlab.com`	Login target for `docker login`
`$CI_REGISTRY_IMAGE`	Full image path for this project	Base image name
`$CI_REGISTRY_USER`	GitLab token username	`docker login -u`
`$CI_REGISTRY_PASSWORD`	Job token	`docker login -p`
`$CI_COMMIT_SHORT_SHA`	8-char commit hash	Image tag
`$CI_COMMIT_BRANCH`	Current branch name	Conditional rules
`$CI_PIPELINE_SOURCE`	`push`, `merge_request_event`, …	Rule filtering
`$CI_MERGE_REQUEST_IID`	MR number	PR-specific artefact naming

Docker-in-Docker (DinD)

Running docker build inside a GitLab CI job requires a Docker daemon. The recommended approach is to use the official docker:dind service — GitLab spins it up as a sidecar container that the job container connects to over TLS.

DinD requires the runner to be configured with the docker executor (or Kubernetes executor with the right settings). It does not work on shell executors by default.

How It Works

┌─────────────────────────────────┐
│         GitLab Runner           │
│                                 │
│  ┌──────────────┐  ┌─────────┐  │
│  │   Job image  │  │  docker │  │
│  │ docker:cli   │◄─│  :dind  │  │
│  │              │  │ daemon  │  │
│  └──────────────┘  └─────────┘  │
└─────────────────────────────────┘
         DOCKER_HOST=tcp://docker:2376

The docker:dind service exposes the Docker socket over TCP (with TLS on port 2376). The job sets DOCKER_HOST to point at it.

Minimal DinD Job

build:image:
  stage: build
  image: docker:cli          # only the CLI — no daemon
  services:
    - docker:dind            # ephemeral Docker daemon as a sidecar
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"   # GitLab mounts certs here automatically
  before_script:
    - docker info            # verify the daemon is reachable
  script:
    - docker build -t my-image:latest .
    - docker run --rm my-image:latest echo "smoke test passed"

Use docker:cli (not docker:latest) as the job image. It ships only the client binary, keeping the image small. The daemon lives exclusively in the docker:dind service container.

Official Docs

GitLab’s services keyword is documented at https://docs.gitlab.com/ci/services/. The services feature isn’t Docker-specific — you can use it to spin up PostgreSQL, Redis, or any other daemon your tests need.

Full Docker Build & Push Pipeline

This is a production-ready template for building a Docker image and pushing it to the built-in GitLab Container Registry on every push to main, plus every MR:

# .gitlab-ci.yml — Docker build + push pipeline
stages:
  - build
  - test
  - release

variables:
  DOCKER_HOST: tcp://docker:2376
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_BASE: $CI_REGISTRY_IMAGE
  IMAGE_TAG: $CI_COMMIT_SHORT_SHA

default:
  image: docker:cli
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

# ── Build ─────────────────────────────────────────────────────────────────────
build:image:
  stage: build
  script:
    - |
      docker build \
        --cache-from $IMAGE_BASE:cache \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --tag $IMAGE_BASE:$IMAGE_TAG \
        --tag $IMAGE_BASE:$CI_COMMIT_BRANCH \
        .
    - docker push $IMAGE_BASE:$IMAGE_TAG
    - docker push $IMAGE_BASE:$CI_COMMIT_BRANCH
  rules:
    - if: $CI_COMMIT_BRANCH
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

# ── Test ──────────────────────────────────────────────────────────────────────
test:smoke:
  stage: test
  script:
    - docker pull $IMAGE_BASE:$IMAGE_TAG
    - docker run --rm $IMAGE_BASE:$IMAGE_TAG /app/healthcheck.sh
  rules:
    - if: $CI_COMMIT_BRANCH
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

# ── Release ───────────────────────────────────────────────────────────────────
release:latest:
  stage: release
  script:
    - docker pull $IMAGE_BASE:$IMAGE_TAG
    - docker tag  $IMAGE_BASE:$IMAGE_TAG $IMAGE_BASE:latest
    - docker push $IMAGE_BASE:latest
    # Update the rolling cache layer
    - docker tag  $IMAGE_BASE:$IMAGE_TAG $IMAGE_BASE:cache
    - docker push $IMAGE_BASE:cache
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Caching & Artifacts

These two features are often confused. The distinction is important:

cache
artifacts
cache + artifacts together

Cache speeds up jobs by persisting files between pipeline runs on the same runner. Use it for dependency directories (node_modules, .venv, Maven’s ~/.m2).

build:node:
  stage: build
  image: node:20-alpine
  cache:
    key:
      files:
        - package-lock.json    # cache busts when lockfile changes
    paths:
      - node_modules/
    policy: pull-push          # pull at job start, push at job end
  script:
    - npm ci
    - npm run build

Cache is not guaranteed to be available — it’s a best-effort optimisation. Never rely on cache for correctness, only for speed.

Artifacts are files explicitly passed between jobs within the same pipeline. They’re stored on the GitLab server and are always reliable.

build:app:
  stage: build
  script:
    - make build
  artifacts:
    paths:
      - dist/                  # available to all downstream jobs
    expire_in: 1 week          # cleaned up automatically

test:integration:
  stage: test
  needs: [build:app]           # download artifacts from this specific job
  script:
    - ls dist/                 # files are here
    - ./run-tests.sh dist/

A common pattern: cache dependencies, artifact the build output.

build:
  stage: build
  image: python:3.12-slim
  cache:
    key: $CI_COMMIT_REF_SLUG
    paths:
      - .venv/
  artifacts:
    paths:
      - dist/
    expire_in: 3 days
  script:
    - python -m venv .venv
    - source .venv/bin/activate
    - pip install -r requirements.txt
    - python -m build

Useful Patterns & Tips

rules vs only/except
needs (DAG)
include & extends

Prefer rules over the legacy only/except keywords — it’s more expressive and evaluates top-to-bottom:

deploy:production:
  stage: deploy
  script: ./deploy.sh prod
  rules:
    - if: $CI_COMMIT_TAG                  # tagged releases only
      when: manual                        # requires a human click
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success                    # auto-deploy on main
    - when: never                         # skip everything else

Use needs to create a Directed Acyclic Graph — jobs start as soon as their dependencies finish, rather than waiting for the entire previous stage:

test:unit:
  stage: test
  needs: [build:app]    # starts as soon as build:app passes

test:e2e:
  stage: test
  needs: [build:app]    # also starts immediately — parallel with test:unit

deploy:staging:
  stage: deploy
  needs:
    - test:unit
    - test:e2e           # waits for both tests, not the whole test stage

Break large pipelines into reusable templates:

# .gitlab/ci/docker.yml — shared template
.docker_job:
  image: docker:cli
  services:
    - docker:dind
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

# .gitlab-ci.yml — main file
include:
  - local: .gitlab/ci/docker.yml

build:image:
  extends: .docker_job     # inherits all keys from the template
  stage: build
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

Docker Essentials

Commands, Dockerfiles, Compose, and networking — the building blocks used inside every pipeline.

Kubernetes

Deploy the images your GitLab pipeline builds into a Kubernetes cluster.

DevOps & CI/CD

Cloud & Infrastructure

Linux & Systems

macOS Tips

GitLab CI/CD: Pipeline Anatomy, Docker-in-Docker & Recipes

`.gitlab-ci.yml` Anatomy

Key Built-in Variables

Docker-in-Docker (DinD)

How It Works

Minimal DinD Job

Official Docs

Full Docker Build & Push Pipeline

Caching & Artifacts

Useful Patterns & Tips

Docker Essentials

Kubernetes

​.gitlab-ci.yml Anatomy

​Key Built-in Variables

​Docker-in-Docker (DinD)

​How It Works

​Minimal DinD Job

​Official Docs

​Full Docker Build & Push Pipeline

​Caching & Artifacts

​Useful Patterns & Tips

​Related Pages

Docker Essentials

Kubernetes

`.gitlab-ci.yml` Anatomy

Key Built-in Variables

Docker-in-Docker (DinD)

How It Works

Minimal DinD Job

Official Docs

Full Docker Build & Push Pipeline

Caching & Artifacts

Useful Patterns & Tips

Related Pages