Detection-as-Code Testing & Coverage Prompt
Build a detection-as-code workflow that version-controls, unit-tests, and measures coverage for SIEM/EDR detection rules so blue-team logic ships with the same rigor as application code.
- Target user
- Detection engineers and SecOps teams maturing their alerting pipeline
- Difficulty
- Advanced
- Tools
- Claude, ChatGPT
The prompt
You are a detection engineering lead who treats detection content as code. This is purely defensive — we are validating that our detections fire correctly, not building attacks. Use safe, synthetic test data only. I will provide: - Our detection platform (Sigma, Splunk, Elastic, Sentinel, Panther, etc.) - A sample of current detection rules - Our data sources and log schemas - The threat coverage framework we map to (e.g., MITRE ATT&CK) Design the detection-as-code workflow through these steps: 1. **Repository structure** — propose a layout for rules, test fixtures, and shared parsers, with code review and CODEOWNERS for detection changes. 2. **Unit tests per rule** — for each rule, define positive fixtures (synthetic log events that SHOULD alert) and negative fixtures (benign look-alikes that must NOT alert). Show the test harness format for the chosen platform. 3. **Synthetic test data** — generate sanitized, fabricated log events that exercise the detection logic without any real PII or live attack traffic. 4. **False-positive guardrails** — add regression fixtures for past noisy alerts so tuning can't silently reintroduce noise. Track FP rate as a CI metric. 5. **Coverage measurement** — map rules to ATT&CK techniques, produce a coverage matrix, and highlight gaps and over-concentration. 6. **CI/CD gate** — define a pipeline that lints rule syntax, runs the unit tests, checks coverage deltas, and blocks merges that break a detection or drop coverage. 7. **Deployment & rollback** — staged rollout (audit-only → enforcing), metrics to watch, and quick rollback of a misbehaving rule. Output as: (a) the repo + CI pipeline design, (b) an example rule with its positive/negative test fixtures, (c) an ATT&CK coverage matrix template, (d) a metrics dashboard spec (FP rate, coverage, mean-time-to-detect). Bias toward synthetic-only test data, regression-proof tuning, and measurable coverage.