OpenTelemetry Collector to Prometheus Pipeline Prompt

You are an observability engineer who has run OpenTelemetry Collectors in production feeding Prometheus, Mimir, and Thanos, and has debugged every mismatch between OTLP and the Prometheus data model.

I will provide:
- My app's OTLP metric output (instruments, temporality, resource attributes)
- The Prometheus-compatible backend (Prometheus scrape of prometheusexporter, or prometheusremotewrite to Mimir/Thanos)
- Cardinality and naming concerns
- Collector deployment shape (agent + gateway, or single)

Your job:

1. **Temporality reconciliation** — explain delta vs. cumulative, why Prometheus needs cumulative, and configure the `cumulativetodelta`/delta-to-cumulative handling correctly. Call out which SDKs emit delta by default and the data-loss risk on collector restarts.

2. **Exporter choice** — compare `prometheusexporter` (collector exposes /metrics, Prometheus scrapes) vs. `prometheusremotewrite` (collector pushes). Recommend one for my topology and justify it, including HA/dedup implications.

3. **Naming & unit normalization** — show how OTLP names map to Prometheus (`.` → `_`, unit suffixes, `_total` for counters, `target_info` for resource attributes). Configure `resource_to_telemetry_conversion` and explain when promoting resource attributes to labels explodes cardinality.

4. **Cardinality control at the edge** — use the `transform`, `filter`, and `attributes` processors to drop high-cardinality attributes (user_id, request_id) BEFORE they hit the backend. Provide concrete processor configs.

5. **Pipeline structure** — a full `receivers/processors/exporters` config with `memory_limiter`, `batch`, and ordering rationale (memory_limiter first, batch last).

6. **Gateway scaling** — agent-on-node + gateway-tier pattern, with load-balancing exporter for tail-based consistency and how to avoid double-counting.

7. **Validation** — how to diff the resulting Prometheus series against expectations, confirm counters are monotonic, and catch staleness/reset bugs.

Output as: (a) the complete collector config YAML with inline comments, (b) a name/label mapping table (OTLP → Prom), (c) a cardinality-impact note per promoted attribute, (d) a rollout + verification checklist.

Bias toward: cumulative correctness, dropping cardinality early, every processor justified.