🚨 Jenkins Error: 'java.lang.StackOverflowError' — Cause, Fix, and Troubleshooting Guide

Overview

A StackOverflowError means a thread exhausted its call stack — the code recursed (or the CPS transform expanded the call chain) deeper than the JVM’s per-thread stack size allows. In Jenkins this shows up either in a pipeline run or in the controller log, with a very long, repeating stack trace:

java.lang.StackOverflowError
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup...
	at com.example.MyLib.buildTree(MyLib.groovy:42)
	at com.example.MyLib.buildTree(MyLib.groovy:47)
	at com.example.MyLib.buildTree(MyLib.groovy:47)
	... repeated hundreds of times ...

The tell-tale sign is the same frame (or a short cycle of frames) repeating. StackOverflowError is an Error, not an Exception, so ordinary catch (Exception) blocks won’t catch it — the build usually dies hard.

Two things make this more common in Jenkins than in plain Java: shared-library recursion, and Pipeline’s CPS (Continuation-Passing Style) transformation, which rewrites your Groovy so each step nests through many more internal frames than the source suggests.

Symptoms

• A build fails with java.lang.StackOverflowError and a huge, repetitive stack trace.
• The failing frames are your own shared-library or pipeline method, called from itself.
• Failure is deterministic for a given input, or scales with input size (a deeper tree / longer list overflows, a small one doesn’t).
• try/catch around the call does not stop the build — an Error isn’t caught.
• On the controller, jenkins.log shows the overflow inside workflow.cps classes for pipeline code, or inside a plugin’s package for a plugin bug.
• Raising heap (-Xmx) makes no difference — this is stack, not heap.

Common Root Causes

• Unbounded or mis-based recursion — a shared-library or pipeline function calls itself with no base case, or a base case that’s never reached (off-by-one, wrong termination condition).
• Mutual recursion — method A calls B calls A, forming a cycle that never terminates.
• CPS deep call chains — the CPS transform adds many internal frames per user call, so even moderately deep legitimate recursion overflows a stack that would be fine in plain Groovy.
• Deeply nested / huge config or data — parsing a deeply nested JSON/YAML/tree structure recursively, where the data depth exceeds the stack budget.
• Plugin defect — a plugin with genuinely recursive logic (serialization, XML/DOM walking) overflows on certain inputs; the trace is inside the plugin, not your code.
• Accidental infinite loop via getters — a Groovy property/toString()/equals() that references itself, causing infinite re-entry.

How to diagnose

Read the stack trace and find the repeating frame — that names the recursive method. The line number tells you the recursive call site:

at com.example.MyLib.buildTree(MyLib.groovy:47)   <-- this line calls itself

Reproduce with a minimal input to see whether depth scales with data. In the Manage Jenkins → Script Console, test a suspect helper in isolation (this runs on the controller — use carefully and read-only):

// Reproduce the recursion outside a build to find the depth where it breaks
int depth = 0
def recurse
recurse = { n -> depth = n; recurse(n + 1) }   // deliberately unbounded
try { recurse(0) } catch (StackOverflowError e) { println "overflowed at depth ${depth}" }

For pipeline code, check whether CPS is amplifying the depth. Search the controller log for the overflow context:

grep -i 'StackOverflowError' /var/jenkins_home/logs/jenkins.log
# Is the trace inside workflow.cps (your pipeline/library) or a plugin package?

Check the current controller thread stack size, since that sets the ceiling:

# Effective JVM flags for the running controller
jcmd $(pgrep -f jenkins.war) VM.flags | tr ' ' '\n' | grep -i thread
java -XX:+PrintFlagsFinal -version | grep -i ThreadStackSize

Fixes

1. Fix the recursion — add or correct the base case

The correct fix is almost always in your code, not the JVM. Ensure every recursive path has a reachable termination condition:

// BAD: no base case reached for empty children
int countNodes(node) {
  return 1 + node.children.sum { countNodes(it) }   // overflows on deep/cyclic trees
}

// GOOD: explicit base case, guarded
int countNodes(node) {
  if (node == null || node.children == null || node.children.isEmpty()) return 1
  int total = 1
  for (child in node.children) total += countNodes(child)
  return total
}

2. Convert recursion to iteration

For deep data, an explicit stack (a List used as a work queue) removes the call-depth ceiling entirely — this is the most robust fix for CPS pipelines, where every call frame is expensive:

// Iterative tree walk — no recursion, no CPS frame explosion
int countNodes(root) {
  int total = 0
  def stack = [root]
  while (!stack.isEmpty()) {
    def node = stack.remove(stack.size() - 1)
    if (node == null) continue
    total++
    if (node.children) stack.addAll(node.children)
  }
  return total
}

3. Move heavy recursion out of CPS with @NonCPS

If the recursion is pure computation (no pipeline steps like sh, git, node), annotate the method @NonCPS so it runs as ordinary Groovy — far fewer frames, and much faster. Note: no pipeline steps are allowed inside a @NonCPS method.

// vars/treeUtils.groovy or a src/ class in the shared library
@NonCPS
int countNodes(root) {
  // plain Groovy recursion here is fine — CPS is bypassed
  if (!root?.children) return 1
  return 1 + root.children.sum { countNodes(it) }
}

4. Increase thread stack size as a stopgap

If the recursion is legitimate and bounded but simply deeper than the default stack allows, raise -Xss. For the controller, add to the JVM args (e.g. JENKINS_JAVA_OPTIONS / systemd override):

-Xss4m        # default is often 512k–1m; raising it buys depth at a memory cost

For build tools launched from a pipeline, raise the tool’s stack, not Jenkins’:

sh 'JAVA_TOOL_OPTIONS="-Xss4m" ./gradlew build'

Treat -Xss as a bandage: larger stacks cost memory per thread, and a truly unbounded recursion will overflow any size. Fix the algorithm too.

5. If it’s a plugin, update or isolate it

When the repeating frames are inside a plugin package, check the plugin’s issue tracker and update to a version with the fix. If none exists, avoid the triggering input/config until it’s patched.

What to watch out for

• StackOverflowError extends Error, not Exception — catch (Exception e) will not catch it. Don’t rely on try/catch to recover.
• Raising -Xmx (heap) does nothing for stack overflow. Only -Xss (per-thread stack) and fixing recursion help.
• In Pipeline, “it worked at the console but overflows in a build” is usually CPS amplification — prefer @NonCPS or iteration for anything recursive.
• Deeply nested Groovy closures and huge parallel maps also grow the stack; flatten them where you can.
• Guard recursive parsers against cyclic input (self-referential trees) which recurse forever regardless of stack size.

Related

• Jenkins Error: NotSerializableException — the other classic CPS pipeline pitfall, caused by holding non-serializable state across steps.
• Jenkins Error: OutOfMemoryError: Java heap space — the heap counterpart; different tuning (-Xmx vs -Xss) and different root causes.
• Jenkins Error: java.lang.NoClassDefFoundError — another JVM-level failure surfaced in pipeline and plugin code.