🚨 Telegraf Error Guide: '[inputs.tail] ... permission denied' — Fix Log File Access

Overview

The tail input follows log files and parses new lines into metrics. When the telegraf service user cannot read a target file, the plugin logs a permission error naming the path:

2026-07-10T12:00:00Z E! [inputs.tail] Error in plugin: open /var/log/nginx/access.log: permission denied

A glob pattern produces the same error per unreadable file, and a locked-down parent directory yields a traversal variant:

E! [inputs.tail] Error in plugin: stat /var/log/app/current.log: permission denied

Telegraf keeps running but produces no metrics from the unreadable file until access is granted.

Symptoms

• Metrics derived from a specific log file are missing while other tail targets work.
• journalctl -u telegraf repeats open ... permission denied for one or more paths.
• Reading the file as root or the app owner works, but sudo -u telegraf cat fails.
• A glob like /var/log/app/*.log matches files but some are silently skipped due to permissions.
• The error appears only after a log-rotation or a deploy that recreated files with tighter modes.

Common Root Causes

• File not readable by telegraf — the log is 0640 root:adm or 0600 app:app and telegraf is in neither group.
• Directory not traversable — a parent directory lacks the execute (x) bit for telegraf, so the file cannot even be opened.
• Group membership missing — the fix is often adding telegraf to adm/syslog/an app group, but the change was not applied or the service not restarted.
• Log rotation recreates files with restrictive modes — logrotate create 0600 app app resets permissions each rotation.
• SELinux or AppArmor denial — mandatory access control blocks Telegraf from /var/log/... even when Unix permissions allow it.
• Symlinked log path — the symlink is readable but its target is not, or the target directory is not traversable.

Diagnostic Workflow

Reproduce the access attempt as the exact service user. This is the definitive test:

sudo -u telegraf cat /var/log/nginx/access.log | head; echo "exit=$?"

Inspect the full permission chain — the file and every parent directory needs the right bits:

namei -l /var/log/nginx/access.log
id telegraf
ls -l /var/log/nginx/access.log

Run only the tail input with debug to confirm which globbed paths fail:

telegraf --config /etc/telegraf/telegraf.conf --test --input-filter tail --debug

A typical tail config:

[[inputs.tail]]
  files = ["/var/log/nginx/access.log"]
  from_beginning = false
  data_format = "grok"
  grok_patterns = ['%{COMBINED_LOG_FORMAT}']

The cleanest fix is group-based read access. Add telegraf to the group that owns the logs and restart:

sudo usermod -aG adm telegraf        # nginx/syslog logs are often group 'adm'
sudo systemctl restart telegraf      # group change requires a restart
sudo -u telegraf cat /var/log/nginx/access.log | head   # verify

If SELinux is enforcing, check for and address AVC denials:

sudo ausearch -m avc -c telegraf --start recent
getenforce

Make log rotation preserve group readability so the fix survives rotation:

# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
    create 0640 www-data adm
}

Example Root Cause Analysis

A tail input on /var/log/app/current.log reported permission denied even though ls -l showed the file as 0644 (world-readable). Running sudo -u telegraf cat still failed. namei -l /var/log/app/current.log revealed the cause: the parent directory /var/log/app was 0750 app:app, and telegraf was not in the app group, so it lacked the execute bit needed to traverse into the directory and open the file at all.

Because the file mode was already world-readable, only directory traversal was missing. Adding telegraf to the app group and restarting the service fixed it:

sudo usermod -aG app telegraf
sudo systemctl restart telegraf

After the restart, sudo -u telegraf cat /var/log/app/current.log succeeded and metrics resumed. The lesson: file permissions are only half the story — every parent directory in the path needs the execute bit for the service user, and namei -l is the fastest way to find the broken link in the chain.

Prevention Best Practices

• Grant read access via group membership (adm, syslog, or an app group) rather than loosening file modes to world-readable.
• Always restart Telegraf after changing group membership; new groups only apply to new processes.
• Verify the entire path with namei -l; a locked parent directory blocks access regardless of file mode.
• Configure logrotate create modes so rotated files stay group-readable and the fix persists.
• On SELinux/AppArmor hosts, add the correct policy or file context instead of disabling enforcement.
• Test every new tail target with sudo -u telegraf cat before deploying the config.

Quick Command Reference

# Definitive access test as the service user
sudo -u telegraf cat /var/log/nginx/access.log | head; echo "exit=$?"

# Show permissions of the file AND every parent directory
namei -l /var/log/nginx/access.log

# Confirm telegraf's group membership
id telegraf

# Run only the tail input with debug
telegraf --config /etc/telegraf/telegraf.conf --test --input-filter tail --debug

# Check for SELinux denials
sudo ausearch -m avc -c telegraf --start recent

# Add telegraf to the log-owning group and restart
sudo usermod -aG adm telegraf && sudo systemctl restart telegraf

Conclusion

[inputs.tail] ... permission denied means the telegraf user cannot read a log file or traverse a directory in its path. Reproduce with sudo -u telegraf cat, walk the full chain with namei -l, and prefer group membership over world-readable modes to grant access. Restart after group changes, make logrotate preserve permissions, and check SELinux/AppArmor on enforcing hosts. Once the service user can read the file, tail metrics flow again.