🚨 Filebeat Error Guide: 'multiline pattern did not match' — Fix Stack Trace Grouping

Overview

When a multiline parser is misconfigured, Filebeat’s debug output shows the pattern failing to match the lines you expected it to join, and each physical line ships as its own event:

multiline: pattern '^\d{4}-\d{2}-\d{2}' did not match line, starting new event: "	at com.example.Service.run(Service.java:42)"

Multiline parsing tells Filebeat how to fold several physical lines (a Java stack trace, a Python traceback, a pretty-printed block) into one logical event. It is driven by three settings: a pattern regex, negate (true/false), and match (before/after). If any of the three is wrong, Filebeat either splits one event into many or glues unrelated lines together. There isn’t always a hard error — the symptom is malformed events downstream — but running with -d reveals the pattern mismatch directly.

Symptoms

• A single stack trace arrives as many separate documents, one per line.
• Continuation lines (at ..., Caused by:, indented frames) appear as standalone events with no context.
• Unrelated log lines get merged into one giant event.
• Debug logs show multiline deciding to “start new event” on lines that should have been appended.
• Alerts and dashboards miss errors because the exception header and its trace are in different documents.

Common Root Causes

• Pattern matches the wrong lines — it should match the start of each event (the timestamp line), not the continuation lines.
• Wrong negate/match combination — the canonical “join lines that don’t start a new event” recipe is negate: true, match: after.
• Unanchored or too-loose regex — missing ^ so it matches mid-line, or .* that matches everything.
• Timestamp format mismatch — the pattern expects YYYY-MM-DD but the log uses MMM DD or epoch.
• Unescaped/unquoted regex in YAML, so the pattern isn’t what you think.

Diagnostic Workflow

Nail down the shape of a real event first, then write a pattern that matches only its first line.

sed -n '1,20p' /var/log/app/app.log     # look at a full multi-line event
grep -nE '^[0-9]{4}-[0-9]{2}-[0-9]{2}' /var/log/app/app.log | head   # verify start-of-event lines

Use the standard “match the start line, append everything after” recipe:

# /etc/filebeat/filebeat.yml
filebeat.inputs:
  - type: filestream
    id: java-app
    paths: ["/var/log/app/app.log"]
    parsers:
      - multiline:
          type: pattern
          pattern: '^\d{4}-\d{2}-\d{2}'   # matches the timestamp that begins each event
          negate: true                     # lines NOT matching...
          match: after                     # ...are appended to the previous event
          max_lines: 500
          timeout: 5s

filebeat test config -c /etc/filebeat/filebeat.yml
filebeat -e -c /etc/filebeat/filebeat.yml -d "input,harvester"   # watch multiline join/split decisions

Example Root Cause Analysis

A team wanted Java stack traces grouped, and wrote the pattern to match the continuation lines instead of the header:

- multiline:
    pattern: '^\s+at '     # BUG: matches only "  at ..." frames
    negate: false
    match: after

With negate: false, match: after, Filebeat appended lines that matched ^\s+at — but the exception header (2026-07-10 ERROR ...) didn’t match, so it started a fresh event, and the first at frame started yet another. Traces were scattered across documents.

The fix was to match the start of each event (the timestamp) and append everything else:

- multiline:
    pattern: '^\d{4}-\d{2}-\d{2}'
    negate: true
    match: after

Now the timestamp line begins each event and all non-timestamp lines (message, at frames, Caused by:) fold into it.

Prevention Best Practices

• Write the pattern to match the first line of an event (usually a timestamp), then use negate: true, match: after.
• Anchor patterns with ^ and single-quote them in YAML so the regex is exactly what you intend.
• Set max_lines and timeout so a runaway trace or a stuck event can’t grow unbounded.
• Test the regex against real sample lines before deploying, and confirm with filebeat -e -d.
• Make sure the timestamp format in the pattern matches the application’s actual log format.

Quick Command Reference

sed -n '1,20p' app.log                    # inspect a full multi-line event
grep -nE '^[0-9]{4}-' app.log              # confirm start-of-event lines
filebeat test config                       # validate multiline config
filebeat -e -d "input,harvester"           # watch join/split decisions
# recipe: pattern '^<timestamp>', negate: true, match: after

Conclusion

A multiline pattern that “did not match” means Filebeat is grouping lines by the wrong rule, so stack traces split into many events or unrelated lines merge. The reliable recipe is to match the line that starts each event — typically the leading timestamp — with negate: true and match: after, so every continuation line folds into it. Anchor and quote the regex, cap it with max_lines/timeout, and verify against real samples with filebeat -e -d before rolling out.