🚨 Linux Error Guide: 'TCP: out of memory -- consider tuning tcp_mem' Socket Memory Pressure

Exact Error Message

When the kernel TCP stack exhausts its global memory pool, it logs warnings to the kernel ring buffer. You will typically see one or more of these lines via dmesg or journalctl -k:

kernel: TCP: out of memory -- consider tuning tcp_mem
kernel: TCP: too many orphaned sockets
kernel: Out of socket memory

These messages are rate-limited, so the count you see is almost never the full picture. A single line can represent thousands of suppressed events. On busy proxies, load balancers, and high-connection API gateways, these are among the most common and most misunderstood kernel networking warnings.

What the Error Means

Every TCP socket on a Linux box draws from a single, system-wide memory budget. That budget is controlled by net.ipv4.tcp_mem, which is three numbers expressed in PAGES, not bytes:

net.ipv4.tcp_mem = low  pressure  high

To convert pages to bytes, multiply by the system page size (almost always 4096 bytes / 4 KiB):

• low watermark — below this, the kernel does not bother reclaiming TCP buffer memory.
• pressure watermark — between pressure and high, the kernel enters “memory pressure” mode and starts trimming send/receive queues and being stingy with buffer growth.
• high watermark — the hard ceiling. When total TCP memory crosses this, the kernel refuses to allocate more buffer space, drops queued data, and logs TCP: out of memory -- consider tuning tcp_mem.

So if tcp_mem = 185565 247423 371130, the high watermark is 371130 * 4096 ≈ 1.52 GB. Once all TCP sockets combined need more than ~1.5 GB of buffer memory, the error fires.

Out of socket memory is a closely related sibling. It fires for two distinct reasons:

1. TCP memory crossed the tcp_mem high watermark (same root cause as above), or
2. The number of orphaned sockets exceeded net.ipv4.tcp_max_orphans.

An orphaned socket is one that is no longer attached to a file descriptor (the application closed it) but is still consuming kernel memory while finishing its TCP shutdown — typically sitting in FIN_WAIT or LAST_ACK. Thousands of these add up fast.

The practical consequence is dropped data: queued bytes that cannot be buffered are discarded, connections stall or reset, and applications see truncated responses, timeouts, or Connection reset by peer.

Common Causes

• Too many concurrent sockets. A connection spike (legitimate traffic or a flood) where each socket’s send/receive buffers sum past the tcp_mem high watermark.
• Oversized per-socket buffers. Aggressive net.ipv4.tcp_rmem / tcp_wmem maximums multiplied across tens of thousands of connections blow the global pool.
• Slow consumers / slow readers. A client (or downstream service) that does not read fast enough forces the kernel to hold large receive queues; a slow network path forces large send queues. Bytes pile up in kernel buffers.
• Orphaned socket storms. Applications opening and closing connections rapidly (short-lived HTTP, health checks, scrapers) leave many sockets in shutdown states, exceeding tcp_max_orphans.
• A tcp_mem ceiling sized for a small box. Defaults are derived from RAM at boot. A container with a tiny memory cgroup, or a VM that was resized, can inherit a low ceiling.

How to Reproduce the Error

You can provoke socket-memory pressure deliberately on a disposable test host. The simplest reproduction is to artificially lower the ceiling and then open many connections that buffer data without being read.

# (test box only) read the current ceiling so you can restore it:
sysctl net.ipv4.tcp_mem

# Lower the high watermark dramatically, then drive thousands of
# connections through a tool like `wrk` or a custom socket flooder
# that writes but never reads. The kernel will cross the lowered
# high watermark and log: TCP: out of memory -- consider tuning tcp_mem

A faster way to see Out of socket memory is to lower tcp_max_orphans to a small value and then rapidly open/close thousands of connections so the orphan count exceeds it. The key insight: you do not need real memory exhaustion, just to cross the configured watermark.

Diagnostic Commands

All of the commands below are read-only and safe to run on a production host.

Check the kernel log for the actual messages and their timing:

dmesg -T | grep -i 'tcp:\|socket memory'
journalctl -k -g 'tcp:'

Read the three relevant sysctl tunables. Remember tcp_mem is in pages; tcp_rmem/tcp_wmem are in bytes (min / default / max):

sysctl net.ipv4.tcp_mem net.ipv4.tcp_rmem net.ipv4.tcp_wmem
sysctl net.ipv4.tcp_max_orphans

The single most useful file is /proc/net/sockstat. The TCP line’s mem field is the current TCP memory usage in pages — compare it directly against the tcp_mem high watermark:

cat /proc/net/sockstat
cat /proc/net/sockstat6

sockets: used 24817
TCP: inuse 23140 orphan 11982 tw 4471 alloc 24001 mem 369880
UDP: inuse 18 mem 12
UDP-LITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

Here mem 369880 pages is ≈ 1.51 GB, sitting right on top of a high watermark of 371130 pages. The orphan 11982 count is also dangerously close to a typical tcp_max_orphans default. This box is on the edge.

Get a global summary and per-socket memory accounting:

ss -s
ss -tm

ss -tm shows the skmem breakdown for each socket. Watch for individual sockets holding large receive (r) or send (t) queues, the fingerprint of a slow consumer:

ESTAB  0  0  10.0.4.21:443  203.0.113.55:51823
     skmem:(r4194304,rb6291456,t0,tb262144,f4096,w0,o0,bl0,d0)

That r4194304 means ~4 MB stuck in this one socket’s receive buffer because the application is not reading fast enough. Multiply that across thousands of connections and the global pool is gone.

Count sockets by state to spot orphan or TIME-WAIT storms:

ss -tan state time-wait | wc -l
ss -tan state fin-wait-1 | wc -l

Step-by-Step Resolution

1. Confirm which limit you hit. Compare mem from /proc/net/sockstat against the tcp_mem high watermark (pages vs pages). If mem is near/over the high number, it is a buffer-pool problem. If orphan is near tcp_max_orphans, it is an orphan problem. Both can be true at once.

2. Right-size tcp_mem if the box genuinely has spare RAM. Raising the ceiling is appropriate when memory is available and the load is legitimate. Set all three watermarks in pages:

   # /etc/sysctl.d/99-tcp-mem.conf  (apply with: sudo sysctl -p ...)
   net.ipv4.tcp_mem = 381300 508400 762600

   That gives a high watermark of 762600 * 4096 ≈ 3.1 GB. Never set this higher than the box can afford alongside application memory, or you trade a TCP warning for an OOM-killer kill.

3. Reduce per-socket buffer maximums if you have huge connection counts. The defaults scale buffers per socket; capping the maximum lowers the multiplied total:

   net.ipv4.tcp_rmem = 4096 131072 6291456
   net.ipv4.tcp_wmem = 4096 16384 4194304

   Lowering the third (max) value bounds how large any one socket can grow, protecting the global pool at the cost of peak single-stream throughput.

4. Raise tcp_max_orphans if you are dropping orphaned sockets and have the RAM (each orphan costs up to ~64 KB):

   net.ipv4.tcp_max_orphans = 262144

5. Fix slow consumers — the real root cause more often than not. Tuning sysctls only buys headroom; a downstream service or client that cannot keep up will refill the buffers. Identify the offending sockets from ss -tm (large r/t queues), then fix the application: add backpressure, increase read concurrency, scale the slow downstream, or shorten timeouts so stuck connections release memory.

6. Recycle TIME-WAIT sockets if a TIME-WAIT storm is consuming the pool. Enabling reuse lets the kernel reclaim sockets for new outbound connections:

   net.ipv4.tcp_tw_reuse = 1

   Do not re-enable the removed tcp_tw_recycle; it breaks connections behind NAT.

7. Persist and validate. Put changes in /etc/sysctl.d/, apply, then re-watch /proc/net/sockstat under load to confirm mem and orphan stay well below their limits.

Prevention and Best Practices

• Alert on mem as a percentage of the tcp_mem high watermark, not just on the kernel log message (which is rate-limited and lossy).
• Size tcp_mem deliberately on resized VMs and small containers — the boot-time defaults may not match the current RAM.
• Keep per-socket buffer maximums modest on high-fan-out hosts (proxies, load balancers) where connection count, not per-stream throughput, dominates.
• Treat slow consumers as bugs. Backpressure in the application is far more durable than ever-larger kernel buffers.
• Watch orphan and TIME-WAIT counts on services that churn many short-lived connections.

For broader kernel-resource tuning, see our Linux admin guides, and for connection-exhaustion symptoms see our guide on TIME-WAIT and ephemeral port exhaustion.

Related Errors

• nf_conntrack: table full, dropping packet — connection tracking table exhaustion, a parallel “too many connections” symptom.
• TCP: request_sock_TCP: Possible SYN flooding — the accept/SYN backlog filling, often alongside socket-memory pressure.
• kernel: Out of memory: Killed process — the OOM killer, which you risk triggering if you over-raise tcp_mem.
• Cannot assign requested address — ephemeral port exhaustion, commonly co-occurring with TIME-WAIT storms.

Frequently Asked Questions

Q: Is tcp_mem measured in bytes or pages? Pages. The three values are in memory pages, and the page size is almost always 4096 bytes. Multiply by 4096 to get bytes. This is the number-one source of confusion: people set tcp_mem to byte values and accidentally configure a multi-terabyte ceiling.

Q: The error appears but my server has plenty of free RAM. Why? Because tcp_mem is an independent ceiling, not tied to free RAM at runtime. You can hit the TCP high watermark with gigabytes of free system memory. Likewise, Out of socket memory can mean you exceeded tcp_max_orphans, which is a count limit, not a byte limit.

Q: Should I just raise tcp_mem to a huge number to make the error go away? No. The warning is a guardrail. Raising it past what the box can afford turns a recoverable TCP drop into an OOM-killer event that kills your application. Raise it only with measured headroom, and fix slow consumers in parallel.

Q: What is an orphaned socket and why does it matter? It is a socket whose application closed it but whose TCP shutdown is still in progress (FIN_WAIT, LAST_ACK). It consumes kernel memory with no file descriptor. Too many at once exceed net.ipv4.tcp_max_orphans and trigger Out of socket memory plus dropped connections.

Q: How do I know if dropped data is happening right now? Watch the mem field in /proc/net/sockstat against the tcp_mem high watermark. When mem reaches the high value, new buffer allocations fail and queued bytes are discarded. Pair that with ss -tm to find the specific sockets holding oversized queues.