🚨 Filebeat Error: 'Failed to start crawler' — Cause, Fix, and Troubleshooting Guide

Overview

The crawler is the Filebeat component that reads your filebeat.inputs (and enabled modules) and spins up a harvester per matching file. If any configured input cannot be initialized, the crawler refuses to start and Filebeat exits:

Exiting: Failed to start crawler: starting input failed: Error while initializing input: No paths were defined for input accessing 'filebeat.inputs.0' (source:'/etc/filebeat/filebeat.yml')

The important detail is in the nested reason after Failed to start crawler:. The crawler aggregates every input’s initialization; a single broken input block — a missing paths, an unknown input type, a bad module config — aborts the whole crawler and stops Filebeat from harvesting anything. The YAML itself is usually valid, so filebeat test config may pass while startup still fails.

Symptoms

• Filebeat exits at boot with Exiting: Failed to start crawler: ....
• The message always carries a nested starting input failed / initializing input reason.
• No harvesters start and no data ships, even though the config “looks” fine.
• The problem appears right after editing an input, enabling a module, or an upgrade.
• filebeat test config reports OK because it validates YAML shape, not full crawler startup.

Common Root Causes

• Input with no paths — a log/filestream input missing its paths: list.
• Unknown or misspelled input type — a value Filebeat does not recognize.
• Broken module config — an enabled module in modules.d/ with an invalid setting.
• Duplicate filestream id — two filestream inputs sharing an id.
• Bad processor attached to an input that fails to construct.

How to diagnose

Run Filebeat in the foreground to see the full nested reason:

filebeat -e -c /etc/filebeat/filebeat.yml

Validate config and list what modules are enabled:

filebeat test config -c /etc/filebeat/filebeat.yml
filebeat modules list

Isolate the suspect input into a minimal config and run only that:

filebeat -e -c /etc/filebeat/one-input.yml -d "crawler,input"

Fixes

Give every input a valid type and a non-empty paths list:

filebeat.inputs:
  - type: filestream
    id: app-logs
    enabled: true
    paths:
      - /var/log/app/*.log

If a module is the culprit, disable it or fix the offending file under modules.d/:

filebeat modules disable nginx        # if nginx.yml is broken
filebeat test config                  # re-validate
systemctl restart filebeat

Ensure each filestream input has a unique id so the crawler does not reject a collision, then restart and confirm harvesters start:

journalctl -u filebeat -f | grep -i 'crawler\|harvester\|Start'

What to watch out for

• Always read the nested reason — Failed to start crawler: is only the wrapper.
• filebeat test config is necessary but not sufficient; do a foreground start before deploying.
• Enabling a module pulls in extra config that can break the crawler even if filebeat.yml is fine.
• Keep configs in version control and validate in CI so a bad input never reaches a running node.

Related

• Filebeat Error: ‘No paths were defined for input’
• Filebeat Error Guide: ‘Failed to create input: unknown input type’
• Filebeat Error Guide: ‘Error creating runner from config’