🚨 Linux Error Guide: 'nf_conntrack: table full, dropping packet' Connection Tracking Exhaustion

Exact Error Message

When the kernel connection tracking table fills up, netfilter starts dropping new packets and logs the event to the kernel ring buffer and syslog. The message is rate-limited, so you typically see a burst rather than one line per dropped packet:

kernel: nf_conntrack: nf_conntrack: table full, dropping packet
kernel: nf_conntrack: nf_conntrack: table full, dropping packet
kernel: nf_conntrack: nf_conntrack: table full, dropping packet
kernel: net_ratelimit: 4063 callbacks suppressed
kernel: nf_conntrack: nf_conntrack: table full, dropping packet
kernel: nf_conntrack: nf_conntrack: table full, dropping packet

The doubled nf_conntrack: nf_conntrack: prefix is normal — it comes from the module name being prepended to the message text. On the application side, the symptom is rarely obvious: connections succeed most of the time but you see intermittent timeouts, hung TCP handshakes, and “connection timed out” errors that come and go in waves:

curl: (28) Failed to connect to api.internal:443 after 30002 ms: Connection timed out
nginx: upstream timed out (110: Connection timed out) while connecting to upstream

Because the drops are silent at the packet level, the same request often succeeds on retry, which makes the failure look flaky and intermittent rather than like a hard outage.

What the Error Means

Linux netfilter (the framework behind iptables and nftables) keeps a connection tracking table so it can correlate packets that belong to the same flow. This is what makes stateful firewalling, NAT, and ESTABLISHED,RELATED rules work. Every tracked flow — one TCP connection, one UDP “conversation”, one ICMP echo pair — occupies one entry in the table.

The table has a hard upper bound, net.netfilter.nf_conntrack_max. When the number of live entries (net.netfilter.nf_conntrack_count) reaches that ceiling, the kernel cannot allocate a slot for a new flow, so it drops the packet that would have started it and logs table full, dropping packet. Existing tracked connections keep working; only new flows are penalized.

Each entry costs roughly 300 bytes of kernel memory, so the table is sized to balance capacity against RAM. The default nf_conntrack_max is derived from system memory and is often as low as 65,536 or 262,144 on modest hosts — easy to exhaust on a busy NAT gateway or Kubernetes node.

Common Causes

• Genuinely high connection volume on NAT gateways, reverse proxies, and Kubernetes nodes (kube-proxy with iptables/IPVS tracks every service connection). The default nf_conntrack_max is simply too small for the workload.
• TIME_WAIT and slow timeouts. After a connection closes, its conntrack entry lingers (default TIME_WAIT tracking is ~120 seconds). High connection churn — short-lived HTTP/1.0 requests, health checks, microservice fan-out — piles up tens of thousands of TIME_WAIT entries that occupy slots long after the traffic is done.
• SYN floods or scanning create huge numbers of half-open SYN_SENT/SYN_RECV entries that fill the table quickly.
• Undersized hashsize/buckets. If nf_conntrack_buckets is too small relative to nf_conntrack_max, hash chains grow long, lookups slow down, and performance degrades even before the table is technically full.
• Tracking traffic that does not need it — bulk backup transfers, DNS, or internal east-west traffic that requires no NAT still consumes entries unless you exempt it with NOTRACK.

How to Reproduce the Error

On a test VM (never in production), shrink the table and generate churn. First lower the ceiling, then open many short connections:

# Temporarily lower the max to make exhaustion easy to trigger (TEST ONLY)
sudo sysctl -w net.netfilter.nf_conntrack_max=512

Then hammer a local service with thousands of brief connections (for example with ab, wrk, or a hping3 SYN burst). Within seconds nf_conntrack_count climbs to 512 and dmesg begins logging table full, dropping packet. Restore the original value afterward with another sysctl -w.

Diagnostic Commands

Start by confirming the kernel is actually logging the error and then compare the live count against the maximum. All of the commands below are read-only and safe on production.

# 1. Confirm the error in the kernel log (with timestamps)
dmesg -T | grep -i conntrack

# 2. Same thing via journald, grepping the kernel ring buffer
journalctl -k -g conntrack

# 3. The key ratio: current entries vs the hard ceiling
sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max

# 4. Read the raw values directly from procfs (same numbers)
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max

# 5. Inspect the hash table size (buckets)
sysctl net.netfilter.nf_conntrack_buckets

If count is at or near max, you have your answer. A healthy host typically sits well below 50% utilization:

net.netfilter.nf_conntrack_count = 261998
net.netfilter.nf_conntrack_max = 262144

That second reading — 261,998 of 262,144 — means the table is effectively full and new flows are being dropped.

Next, find out what is filling it. The conntrack tool (from the conntrack-tools package) lists and summarizes entries:

# Per-CPU stats: look at the "insert_failed" and "drop" counters
conntrack -S

# List all tracked flows (read-only dump)
conntrack -L

# Count entries grouped by TCP state to see TIME_WAIT pileup
conntrack -L | awk '{print $4}' | sort | uniq -c | sort -rn

A table dominated by TIME_WAIT is the classic churn signature:

 198432 TIME_WAIT
  41207 ESTABLISHED
  18044 SYN_SENT
   3120 CLOSE_WAIT

Finally, check the timeout that controls how long those TIME_WAIT entries stick around:

sysctl net.netfilter.nf_conntrack_tcp_timeout_time_wait

net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120

Step-by-Step Resolution

1. Get immediate relief by raising the ceiling. If the host has spare RAM, raise nf_conntrack_max. Remember the cost: ~300 bytes per entry, so 1,000,000 entries ≈ 300 MB of kernel memory. Apply it live and persist it:

# Live (lost on reboot)
sysctl -w net.netfilter.nf_conntrack_max=1048576

# Persist in /etc/sysctl.d/99-conntrack.conf
net.netfilter.nf_conntrack_max = 1048576

2. Grow the hash table to match. Raising max without enlarging the hash (buckets/hashsize) leaves long hash chains and slow lookups. The rule of thumb is nf_conntrack_max = 8 * hashsize. Set the hashsize via the module parameter in /sys:

# So for max=1048576, hashsize = 1048576 / 8 = 131072
echo 131072 > /sys/module/nf_conntrack/parameters/hashsize

# Persist for boot by setting the module option:
# /etc/modprobe.d/nf_conntrack.conf
options nf_conntrack hashsize=131072

3. Shrink the table by lowering timeouts. Raising max treats the symptom; lowering timeouts attacks the cause when TIME_WAIT dominates. Cutting the TIME_WAIT tracking timeout frees slots much faster:

# /etc/sysctl.d/99-conntrack.conf
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 30
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 30
# Also help reuse sockets at the TCP layer:
net.ipv4.tcp_tw_reuse = 1

4. Stop tracking traffic that does not need it. For high-volume flows that require no NAT or stateful filtering, mark them NOTRACK in the raw table so they never consume an entry:

# Don't track DNS to/from a trusted internal resolver (iptables example)
iptables -t raw -A PREROUTING -p udp --dport 53 -j NOTRACK
iptables -t raw -A OUTPUT     -p udp --sport 53 -j NOTRACK

5. Re-measure. Re-run sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max and conntrack -S. The count should now sit comfortably below max with insert_failed no longer incrementing.

Prevention and Best Practices

• Alert on utilization, not just the log. Graph nf_conntrack_count / nf_conntrack_max and page when it exceeds ~80%. Waiting for the dmesg line means you are already dropping packets. See our guide on building a monitoring strategy for thresholds.
• Size for peak churn. Estimate peak connections-per-second × average entry lifetime. A proxy doing 5,000 conn/s with 120 s TIME_WAIT needs room for 600,000 entries before headroom.
• Keep max and hashsize in lockstep at the 8:1 ratio whenever you tune either one.
• Lower timeouts before raising max. Memory is finite; shrinking how long dead flows linger is often cheaper than buying capacity for them.
• Exempt high-volume, non-NAT traffic with NOTRACK rather than letting it crowd out flows that genuinely need tracking.
• On Kubernetes nodes, set conntrack limits via kubelet (--conntrack-max-per-core, --conntrack-tcp-timeout-*) and via a tuned sysctl DaemonSet so every node is consistent.

Related Errors

• nf_conntrack: falling back to vmalloc — the hash table grew large enough to need non-contiguous memory; informational, not fatal.
• net_ratelimit: N callbacks suppressed — accompanies the conntrack drops and tells you how many log lines were throttled.
• TCP: out of memory -- consider tuning tcp_mem — a different exhaustion (socket buffers), but often appears alongside conntrack pressure under load.
• nf_conntrack: automatic helper assignment is deprecated — unrelated config warning about conntrack helpers, frequently seen in the same logs.

Frequently Asked Questions

Q: Will raising nf_conntrack_max ever hurt me? A: Only through memory. Each entry is roughly 300 bytes, so a million-entry table costs about 300 MB of unswappable kernel memory. On a host with plenty of RAM that is fine; on a small VM, raising the ceiling too far can pressure the kernel allocator. Size it to your real peak connection count plus headroom, not to the largest number you can type.

Q: Why is my table full when I have far fewer active connections than that? A: Closed connections do not vanish immediately — they sit in TIME_WAIT (and similar states) for the configured timeout, often 120 seconds. Run conntrack -L | awk '{print $4}' | sort | uniq -c and if TIME_WAIT dominates, lower nf_conntrack_tcp_timeout_time_wait rather than just raising max.

Q: What is the relationship between nf_conntrack_max and nf_conntrack_buckets/hashsize? A: buckets (hashsize) is the number of hash slots; max is the total entries allowed. The kernel default and general rule of thumb is max = 8 * hashsize. Raising max without raising hashsize leaves chains of ~8+ entries per bucket, slowing lookups. Keep them proportional.

Q: Can I just disable connection tracking entirely? A: Not globally if you rely on NAT or stateful firewall rules — those features depend on conntrack. You can selectively exempt specific traffic with NOTRACK rules in the raw table for flows that need neither NAT nor stateful inspection (bulk transfers, trusted internal DNS), which removes them from the table without breaking the rest.

Q: A SYN flood is filling my table — what helps? A: Half-open connections from a SYN flood create many SYN_RECV entries. Enable SYN cookies (net.ipv4.tcp_syncookies = 1), shorten nf_conntrack_tcp_timeout_syn_recv, and rate-limit new SYNs at the firewall so malicious half-opens cannot consume the whole table.