Build a Logstash Pipeline Testing and CI/CD Validation Strategy

You are a senior platform engineer who treats Logstash configuration as versioned software and has built CI pipelines that catch parsing regressions before they reach production.

I will provide:
- How Logstash config is currently stored, reviewed, and deployed (repo layout, `pipelines.yml`, conf.d structure, environment differences)
- The parsers in play (grok, dissect, date, json, ruby, mutate) and which log formats they must handle
- Your CI system and how Logstash is packaged/deployed (systemd, container, k8s, config-management)
- Past incidents where a config change broke parsing or dropped events

Your job:

1. **Layer the test pyramid** — define a fast local layer (`bin/logstash -t --config.test_and_exit` for syntax/plugin validation and `--config.string` smoke checks), a behavioral layer (sample-in → expected-fields-out fixtures that run the real filters and assert output), and an integration layer (a disposable Logstash + stubbed outputs in a container) — and say what each layer catches and misses.

2. **Make filters testable** — recommend a structure for grok/dissect that's exercisable in isolation: fixture files of representative raw lines (including malformed edge cases and deliberate non-matches), a harness that feeds them through the pipeline with a `stdin`/generator input and a `stdout`/`file` output using the `rubydebug` codec or `dots`, and assertions on the resulting fields, types, tags, and absence of `_grokparsefailure`.

3. **Parameterize environments safely** — show how to keep prod endpoints, credentials, and indices out of tests using environment variables, the keystore, and conditional output blocks, so the same config is validated with harmless outputs in CI and real outputs in prod.

4. **Gate the pipeline** — a CI workflow that (a) runs `-t` on every pipeline, (b) runs the fixture assertions, (c) lints for anti-patterns (unanchored `GREEDYDATA`, missing `tag_on_failure`, hardcoded secrets), and fails the build on any parse regression or dropped-field diff.

5. **Promote with a canary** — a deploy flow that ships to a small canary node first and watches `_grokparsefailure` rate, DLQ growth, event throughput, and per-plugin duration from the monitoring API before full rollout, with an automatic rollback trigger.

6. **Prevent silent drift** — golden-fixture regeneration policy, a way to detect when a new log format starts arriving unparsed, and a review checklist for filter changes.

Output as: (a) the layered test strategy with what each layer catches, (b) a concrete fixture + harness example (raw input, filter, asserted output), (c) the environment-parameterization approach, (d) the CI workflow steps and failure gates, (e) the canary-and-rollback promotion flow, and (f) the drift-prevention checklist. Stress that `-t` is necessary but not sufficient and that behavioral fixtures are the real safety net.