Logstash Error: 'Pipeline worker error, the pipeline will be

Overview

Logstash runs each pipeline on a pool of worker threads. Every event is processed on a worker as it passes through the filter and output stages. If a filter or output plugin raises an exception that nothing catches, the worker thread dies — and because a dead worker means the pipeline can no longer process events reliably, Logstash tears the whole pipeline down. The failure looks like this in /var/log/logstash/logstash-plain.log:

[ERROR][logstash.javapipeline ][main] Pipeline worker error, the pipeline will be stopped {:pipeline_id=>"main", :error=>"(NoMethodError) undefined method `strip' for nil:NilClass", :exception=>Java::OrgJrubyExceptions::NoMethodError, :backtrace=>["(ruby filter code):5:in `block in filter_method'", "..."], :thread=>"#<Thread:0x1a2b3c run>"}

This is fundamentally different from a per-event failure tag like _grokparsefailure or _rubyexception. A tag marks one bad event and the pipeline keeps running; a worker error is an unhandled exception that halts ingestion for the entire pipeline. The two fields that matter are :error (the exception message) and :backtrace (where it was raised) — together they name the exact plugin and line responsible.

Symptoms

The pipeline stops and stays stopped; curl -s localhost:9600/_node/stats/pipelines?pretty shows the pipeline missing or not processing events.
A single Pipeline worker error, the pipeline will be stopped line, followed by the pipeline shutting down — not a stream of per-event warnings.
Ingestion halts entirely: nothing new reaches the outputs, and upstream queues (Beats, persistent queue) begin to back up.
The :error names a Ruby/Java exception such as NoMethodError, NoMethodError: undefined method 'strip' for nil:NilClass, TypeError, or an off-heap/OutOfMemoryError.
The :backtrace points at (ruby filter code) with a line number, or at a specific plugin’s source file.

Common Root Causes

A ruby filter dereferencing nil — code like event.get('name').strip blows up when the name field is absent on some events, raising NoMethodError ... for nil:NilClass.
A plugin bug or version mismatch — a filter or output plugin throws on an input shape it does not handle; upgrading or downgrading the plugin is the real fix.
Off-heap / OOM pressure — a worker dies with an OutOfMemoryError when the JVM heap is exhausted, which surfaces here as a worker crash rather than a clean heap error.
A malformed event reaching a strict plugin — an output or codec that assumes a field type raises when it receives something unexpected.
An exception raised in a custom script/codec — anything that runs on the worker thread and does not guard its own failure paths.

How to diagnose

Start with the log line itself. The :error and :backtrace tell you almost everything — read them before touching config:

sudo grep -A3 'Pipeline worker error' /var/log/logstash/logstash-plain.log | tail -n 20

If the backtrace names (ruby filter code):<N>, open the offending ruby filter and go straight to that line. Logstash .conf files use a Ruby-like DSL, so pipeline config is shown as ruby:

filter {
  ruby {
    # line 5 here raises when [name] is missing on an event:
    code => 'event.set("clean_name", event.get("name").strip)'
  }
}

Confirm whether this is memory pressure instead. If the :error mentions OutOfMemoryError, check heap and GC before blaming a filter:

curl -s localhost:9600/_node/stats/jvm?pretty | grep -E 'heap_used_percent|heap_max'
sudo grep -Ei 'OutOfMemoryError|GC overhead' /var/log/logstash/logstash-plain.log

Reproduce in isolation with a minimal config and stdin, so you can feed the exact event shape that triggered the crash:

echo '{"other":"value"}' | sudo -u logstash /usr/share/logstash/bin/logstash \
  -f /etc/logstash/conf.d/suspect.conf --path.settings /etc/logstash

If a plugin (not your own ruby code) is in the backtrace, check its installed version:

sudo /usr/share/logstash/bin/logstash-plugin list --verbose | grep -i <plugin-name>

Fixes

Guard ruby filter code against nil so a missing field can never crash the worker. Use the safe-navigation operator or an explicit check, and always leave a sane default:

filter {
  ruby {
    code => '
      name = event.get("name")
      event.set("clean_name", name.nil? ? "" : name.strip)
    '
  }
}

The same intent with safe navigation:

filter {
  ruby {
    code => 'event.set("clean_name", event.get("name")&.strip || "")'
  }
}

If the backtrace points at a plugin rather than your code, upgrade it to a version that handles the input:

sudo /usr/share/logstash/bin/logstash-plugin update logstash-filter-<name>

If the :error is an OutOfMemoryError, this is heap pressure surfacing as a worker crash — raise the heap in jvm.options (keep -Xms and -Xmx equal) and see the dedicated Java heap space guide:

-Xms2g
-Xmx2g

Finally, split unrelated data streams into separate pipelines in pipelines.yml so one faulty filter cannot take down all ingestion:

- pipeline.id: app-logs
  path.config: "/etc/logstash/conf.d/app.conf"
- pipeline.id: audit-logs
  path.config: "/etc/logstash/conf.d/audit.conf"

What to watch out for

A passing --config.test_and_exit does not catch this — the config is syntactically valid; the exception only fires when a real event hits the bad line at runtime.
Never assume a field is present. Any event.get(...) in a ruby filter can return nil for some subset of events, so guard every dereference.
Isolating streams into separate pipelines contains the blast radius but does not fix the bug — a crashing pipeline still stops processing its own events until you address the root cause.
Watch for it recurring after a plugin upgrade or a change in upstream event shape; add an alert on the Pipeline worker error log pattern so a stopped pipeline is noticed immediately rather than discovered as a gap in your indices.

Logstash Error: 'Pipeline worker error, the pipeline will be stopped' — Cause, Fix, and Troubleshooting Guide

Fixing errors like this? Get 500 free DevOps AI prompts