🚨 Linux Error: Too many open files — Cause, Fix, and Troubleshooting Guide

Summary

Too many open files maps to two related errnos. EMFILE (24) means a single process hit its own open-file-descriptor limit; ENFILE (23) means the whole system ran out of file-table entries. Both print the same string. Because Linux counts sockets, pipes, event fds, and inotify instances as descriptors, the limit is reached by far more than disk files — a busy server counts every socket against the same ceiling. The relevant limit is the per-process soft ulimit, the systemd unit’s LimitNOFILE, or the system-wide fs.file-max.

Common Symptoms

• A service logs accept/socket/open failures with “Too many open files” under load.
• Errors like nginx: [crit] accept4() failed (24: Too many open files) or java.io.IOException: Too many open files.
• New connections are refused while existing ones keep working.
• The open-fd count for a PID climbs steadily and never drops (a leak).
• fs.file-nr shows allocated descriptors approaching fs.file-max.

Most Likely Causes of the ‘Too many open files’ Error

The Too many open files error is driven by these causes, most common in production first:

1. Per-process soft ulimit -n too low — a default of 1024 is trivially exceeded by any connection-heavy daemon.
2. A file-descriptor or socket leak in the application — fds (often sockets stuck in CLOSE_WAIT) climb without bound until they hit the ceiling, independent of load.
3. A systemd service with no or low LimitNOFILE — /etc/security/limits.conf does not apply to systemd services, so the unit inherits a low default.
4. The hard limit blocks raising the soft limit — a process can only raise its soft limit up to the hard limit.
5. System-wide fs.file-max / fs.nr_open exhausted — the kernel caps total allocated descriptors, producing ENFILE for every process.
6. inotify watch/instance exhaustion — fs.inotify.max_user_watches or max_user_instances runs out (IDEs, log shippers, tail -f at scale) and fails with the same string.

Quick Triage

# Pick the offending process and read its live fd count vs limit
PID=$(pgrep -o nginx)
ls /proc/$PID/fd | wc -l
cat /proc/$PID/limits | grep 'open files'

# Host-wide: allocated / unused / max descriptors
cat /proc/sys/fs/file-nr

If the fd count sits at exactly the soft limit, it is a per-process EMFILE. If file-nr field 1 is near file-max, the whole host is out (ENFILE).

Diagnostic Commands

# Effective per-process limits (the ones actually enforced, not your shell's)
sudo prlimit --pid $PID --nofile

Shows both soft and hard NOFILE. If both are low, you must raise the hard limit before the soft one can grow.

# System-wide ceilings
sysctl fs.file-max fs.nr_open

fs.file-max caps total descriptors host-wide; fs.nr_open caps a single process.

# Limit vs leak: sample the fd count under steady load
for i in 1 2 3 4 5; do ls /proc/$PID/fd | wc -l; sleep 30; done

# What is the process actually holding open?
ls -l /proc/$PID/fd | awk '{print $NF}' | sort | uniq -c | sort -rn | head
ss -tanp 2>/dev/null | grep "pid=$PID" | awk '{print $1}' | sort | uniq -c | sort -rn

A steadily climbing count under constant load is a leak; a count pinned at the limit under heavy load is a low ceiling. A pile of CLOSE_WAIT sockets or thousands of handles to the same file points at a leak in the code.

# systemd unit limit and inotify ceilings
systemctl show <SERVICE> -p LimitNOFILE
sysctl fs.inotify.max_user_watches fs.inotify.max_user_instances

Commands are the same on Ubuntu/Debian and RHEL/Rocky; only unit-file paths (/lib/systemd/system vs /usr/lib/systemd/system) differ.

Fix / Remediation

1. Raise the per-process limit live (immediate mitigation to keep the service up):

   sudo prlimit --pid $PID --nofile=65535:65535

2. Set LimitNOFILE on the systemd unit so the limit persists across restarts:

   sudo systemctl edit <SERVICE>     # add:  [Service]\nLimitNOFILE=65535
   sudo systemctl daemon-reload
   sudo systemctl restart <SERVICE>

   For login sessions (non-systemd), set nofile in /etc/security/limits.d/.
3. Raise the system-wide ceiling if you hit ENFILE, persistently:

   sudo sysctl -w fs.file-max=2097152
   echo 'fs.file-max = 2097152' | sudo tee /etc/sysctl.d/99-files.conf

4. Raise inotify watches ahead of file watchers/log shippers:

   sudo sysctl -w fs.inotify.max_user_watches=524288
   echo 'fs.inotify.max_user_watches = 524288' | sudo tee /etc/sysctl.d/99-inotify.conf

5. If it is a leak, fix the code — close sockets, files, and HTTP responses (try-with-resources, defer Close(), context managers). Raising limits only delays the crash.

Warning: systemctl restart drops every in-flight connection the service is handling. Do it in a maintenance window, or drain the node behind a load balancer first.

Validation

# New limit is live after restart
cat /proc/$(pgrep -o <SERVICE>)/limits | grep 'open files'

# fd count is flat under steady load (leak fixed)
for i in 1 2 3; do ls /proc/$(pgrep -o <SERVICE>)/fd | wc -l; sleep 60; done

# Host-wide headroom restored
cat /proc/sys/fs/file-nr

A limit that reflects the new value, a flat fd count under load, and file-nr well below file-max confirm the fix.

Prevention

• For systemd services set LimitNOFILE= explicitly in the unit — limits.conf does not apply, and the inherited default is usually too low.
• Monitor /proc/<pid>/fd count per critical service and alert at ~80% of its soft limit so a leak is caught before it errors.
• Track /proc/sys/fs/file-nr against fs.file-max host-wide to catch ENFILE early.
• Always close sockets, files, and HTTP responses in application code — most fd exhaustion is a leak, not a small limit.
• Raise fs.inotify.max_user_watches before deploying log shippers or watchers that tail many paths.
• Set fs.file-max, fs.nr_open, inotify limits, and LimitNOFILE persistently via /etc/sysctl.d/ and config management (Ansible/Puppet) so they survive reboots.

Related Errors

• Linux Error: fork: Resource temporarily unavailable
• Linux Error: nf_conntrack table full
• Linux Error: Cannot allocate memory
• Linux Error: bind: Address already in use
• Linux Error: TCP out of memory
• All Linux error guides

Final Notes

Too many open files is EMFILE (one process hit its own fd limit) or ENFILE (the host ran out of file-table slots) — the same string covers both. Read /proc/<pid>/limits and /proc/<pid>/fd, compare against fs.file-nr, and raise the limit that actually applies. But if the count keeps climbing under steady load, fix the leak in the code rather than chasing the ceiling.

Want faster Linux incident response? Use DevOps AI Toolkit to turn production errors into clear diagnostics, remediation steps, and reusable runbooks.