📈 Monitor Every Docker Workload From One Observability Stack

The Docker Monitoring and Alerting Bundle is a self-hosted observability stack for small infrastructure teams. It runs entirely on Docker Compose v2, ships with pinned images, and lets you pick one of two Prometheus-API-compatible metrics backends — Prometheus or VictoriaMetrics — without changing any dashboard or alert rule.

This document explains the components, how they fit together, the profile matrix, the network and exposure model, and where data and config live. For install steps see installation.md; for tuning see configuration.md.

Design goals

• Dual backend, one config. Both backends speak PromQL and the Prometheus query API. The Grafana datasource is templated ($METRICS_DS_URL) behind a stable datasource uid: metrics, so every dashboard and every alert rule works unchanged on either backend.
• Shared scrape + rules. VictoriaMetrics scrapes the same config/prometheus/prometheus.yml via its -promscrape.config flag, and vmalert evaluates the same alerts/**/*.yml rules that Prometheus does. You never maintain two copies.
• Opt-in surface area. Compose profiles let you run only what you need: core + a backend, plus optional logs, blackbox probing, and a GPU exporter.
• Safe by default. Every published port binds to ${BIND_ADDR:-127.0.0.1}. Every long-running service is health-checked. Every image tag is pinned.

Components

Only one of prometheus / victoriametrics runs at a time — they are two implementations of the same job.

Data flow

                         scrape (pull, 30s)
   node-exporter :9100  ─────────────┐
   cAdvisor      :8080  ─────────────┤
   blackbox      :9115  ─────────────┤        ┌──────────────────────────┐
   dcgm-exporter :9400  ─────────────┼───────▶│  METRICS BACKEND         │
   edition exporters    ─────────────┘        │  Prometheus :9090        │
                                              │      OR                  │
                                              │  VictoriaMetrics :8428   │
                                              └───────┬──────────┬───────┘
                                                      │          │
                              PromQL query (uid=metrics)         │ rule eval
                                                      │          │ (Prometheus
                                              ┌───────▼──────┐   │  internal, or
                                              │  Grafana     │   │  vmalert :8880)
                                              │  :3000       │   │
                                              └───────┬──────┘   ▼
                                                      │   ┌──────────────┐
                              LogQL query (uid=logs)  │   │ Alertmanager │
                                              ┌───────▼─┐ │ :9093        │
   containers/host logs ──push──▶ Loki :3100 ─┤ (dash-  │ └──────┬───────┘
        via Alloy                             │  boards)│        │ route/group/
                                              └─────────┘        │ inhibit
                                                                 ▼
                                         email · Slack · Teams · webhook
                                         (+ Discord/PagerDuty/Opsgenie)

• Scrape → store: The backend pulls metrics from each exporter every 30s (global.scrape_interval) and stores them locally in a TSDB.
• Store → Grafana: Grafana queries the backend through the metrics datasource. Because the datasource type is prometheus and VictoriaMetrics is Prometheus-API compatible, dashboards do not know or care which backend answers.
• Store → alerts: With the Prometheus backend, Prometheus evaluates the rule files itself. With the VictoriaMetrics backend, vmalert evaluates the same rule files against VictoriaMetrics. Either way, firing alerts are pushed to Alertmanager, which groups, inhibits, and routes them to channels.
• Logs (optional): Alloy discovers Docker containers and tails host syslog, then pushes to Loki. Grafana queries Loki through the logs datasource.

Profile matrix

Profiles are Compose's mechanism for opt-in services. You always run core plus exactly one backend, and add the rest as needed.

# Prometheus backend, core only
docker compose --profile core --profile prometheus up -d

# VictoriaMetrics backend, plus logs and blackbox
docker compose --profile core --profile victoriametrics --profile logs --profile blackbox up -d

scripts/install.sh selects the right profiles from METRICS_BACKEND and the --enable-logs / edition flags, so you rarely type these by hand.

Editions

The bundle ships six editions — opinionated target/alert/dashboard bundles for a common shape of infrastructure:

single-docker-host · docker-compose-application · gitlab-runners · web-hosting-server · gpu-ai-workloads · small-openstack

An edition never edits vendor files. It registers exporter targets under config/prometheus/targets/edition-*.yml (file_sd, auto-reloaded) and its alert rules live under the matching alerts/<category>/ directory. See configuration.md for enabling editions.

Ports

All published ports bind to ${BIND_ADDR} (default 127.0.0.1). Internal-only services are not published at all — they are reached over the monitoring Docker network by service name.

Ports are configurable via *_PORT variables in .env. See security.md before changing BIND_ADDR.

Network model

• A single user-defined bridge network, ${COMPOSE_PROJECT_NAME:-docker-monitoring}_net, connects every service. Services address each other by name (prometheus:9090, loki:3100, blackbox-exporter:9115).
• Only the published services (table above) are reachable from the host, and only on BIND_ADDR. Exporters stay entirely inside the network.
• To reach Grafana from a workstation, use an SSH tunnel or a reverse proxy you control — do not flip BIND_ADDR to 0.0.0.0 without a firewall/VPN in front. See security.md.

Where data and config live

Named volumes (managed by Docker, survive down/up, captured by backup.sh):

• prometheus_data → /prometheus (TSDB)
• victoriametrics_data → /victoria-metrics-data
• grafana_data → /var/lib/grafana (dashboards state, users, org)
• alertmanager_data → /alertmanager (silences, notification log)
• loki_data → /loki (log chunks + index)
• alloy_data → /var/lib/alloy/data (positions)

Bind mounts (your config, read-only into containers, version-controlled):

• ./config/grafana/provisioning → datasource + dashboard providers
• ./dashboards → dashboard JSON, mounted read-only into Grafana
• ./config/prometheus/prometheus.yml → shared scrape/rules config
• ./config/prometheus/targets/ → file_sd targets (auto-reload)
• ./alerts/ → alert + recording rules
• ./config/alertmanager/alertmanager.yml → routing
• ./config/loki/loki-config.yml, ./config/blackbox/blackbox.yml, ./config/alloy/config.alloy

Host mounts for collection (read-only): node-exporter mounts /, cAdvisor mounts /, /sys, /var/run, /var/lib/docker, and Alloy mounts /var/log, /var/lib/docker/containers, and the Docker socket. The security implications of the socket and host mounts are covered in security.md.

Shared building blocks

The compose file factors out three anchors applied across services:

• Logging — json-file with max-size: 10m, max-file: 3 (bounded log growth).
• Restart — ${RESTART_POLICY:-unless-stopped}.
• Security — no-new-privileges:true on every service that can accept it. cAdvisor, Alloy, and the DCGM exporter need extra host access and are documented as exceptions in security.md.