Logstash Error: 'Timeout executing grok'

Overview

The grok filter compiles your patterns into a regular expression and runs it against a field. To protect the pipeline, it aborts any single match that runs longer than timeout_millis (default 30000 ms) and tags the event _groktimeout:

[WARN ][logstash.filters.grok][main] Timeout executing grok '%{GREEDYDATA:msg}' against field 'message' with value 'Value too large to output (2048 bytes)! First 255 chars are: ...'!

The value being too large to print is the tell: this is catastrophic backtracking. Greedy, unanchored patterns — especially multiple %{GREEDYDATA} or .* segments — against a long line make the regex engine explore an exponential number of match paths. A pattern that is instant on a 40-character line can take minutes on a 2 KB line. The event is not malformed; the pattern is pathological. Every timeout burns a worker thread for up to timeout_millis, so a burst of long lines can stall the whole pipeline.

Symptoms

Timeout executing grok ... against field 'message' warnings, often with Value too large to output.
Events tagged _groktimeout (distinct from _grokparsefailure, which means “no match”, not “too slow”).
Pipeline throughput collapses when certain long or unusual log lines arrive; CPU spikes on grok workers.
The same config is fast in the Grok Debugger on short samples but times out in production on real, long lines.
Multiple %{GREEDYDATA} or .* tokens appear in the pattern, or the pattern is not anchored to line start/end.

Common Root Causes

Multiple greedy tokens — two or more %{GREEDYDATA} / .* in one pattern create ambiguous split points and exponential backtracking.
Unanchored patterns — without ^ and $, the engine tries to match starting at every position in a long line.
Regex against fixed-delimiter logs — using grok where the format is a simple, delimited structure that dissect would parse deterministically.
Very long input lines — stack traces, serialized payloads, or base64 blobs in message amplify any backtracking.
timeout_millis left at 30s — a single bad line ties up a worker for 30 seconds before it gives up.

How to diagnose

Confirm it is a timeout, not a plain parse failure, and find which lines trigger it:

sudo tail -f /var/log/logstash/logstash-plain.log | grep -Ei 'Timeout executing grok|_groktimeout'

Check the current filter for the classic backtracking shape — more than one greedy token and no anchors:

# BAD: two greedy tokens, unanchored — catastrophic on long lines
filter {
  grok {
    match => { "message" => "%{GREEDYDATA:head} ERROR %{GREEDYDATA:msg}" }
  }
}

Reproduce cheaply in the Kibana Grok Debugger (Dev Tools → Grok Debugger) using a real long line, not a short sample — the pathology only shows on length. To isolate which pipeline is affected, watch worker utilization via the monitoring API:

curl -s localhost:9600/_node/stats/pipelines?pretty | grep -A6 grok

If you cannot immediately rewrite the pattern, lower the timeout so a bad line fails fast instead of pinning a worker for 30 seconds:

filter {
  grok {
    match         => { "message" => "%{GREEDYDATA:head} ERROR %{GREEDYDATA:msg}" }
    timeout_millis => 5000
    timeout_scope => "event"
  }
}

Fixes

The durable fix is to remove the backtracking. Anchor the pattern and replace the second greedy token with a specific one:

# GOOD: anchored, single greedy tail, structured leading fields
filter {
  grok {
    match => {
      "message" => "^%{TIMESTAMP_ISO8601:ts} \[%{LOGLEVEL:level}\] %{DATA:logger} - %{GREEDYDATA:msg}$"
    }
    break_on_match => true
    timeout_millis => 5000
    timeout_scope  => "event"
  }
}

For fixed-delimiter logs, drop grok entirely and use dissect. It splits on literal delimiters with no regex and no backtracking, so it cannot time out:

filter {
  dissect {
    mapping => {
      "message" => "%{ts} [%{level}] %{logger} - %{msg}"
    }
  }
}

Use dissect to carve off the stable prefix, then a small anchored grok only on the variable tail — this keeps regex work bounded:

filter {
  dissect {
    mapping => { "message" => "%{ts} [%{level}] %{+msg}" }
  }
  grok {
    match          => { "level" => "^%{LOGLEVEL:level}$" }
    break_on_match => true
    timeout_millis => 3000
  }
}

After deploying, confirm the timeouts stop and no events carry _groktimeout:

sudo systemctl restart logstash
sudo tail -f /var/log/logstash/logstash-plain.log | grep -Ei 'groktimeout|Timeout executing grok'

What to watch out for

_groktimeout and _grokparsefailure are different problems: one is too slow, the other is no match. Tune for the one you actually see.
Never chain multiple %{GREEDYDATA} in a single pattern — that is the single biggest cause of backtracking blowups.
Always anchor with ^ and $; an unanchored pattern retries from every offset in a long line.
Prefer dissect for any log with a stable delimiter layout — it is faster and cannot time out, unlike regex.
Test with production-length lines, not short samples; the pathology is a function of input length and only appears at scale.
Lowering timeout_millis limits blast radius but does not fix a bad pattern — fix the regex, then keep a short timeout as a guardrail.

Logstash Error: 'Timeout executing grok' — Cause, Fix, and Troubleshooting Guide

Fixing errors like this? Get 500 free DevOps AI prompts