systemd-coredump + eu-unstrip Crash Analysis
Capture and analyze userspace crashes with systemd-coredump and coredumpctl — find the faulting frame, match build-id debug symbols with eu-unstrip, and catch the recurring crasher.
- Target user
- Linux admins triaging crashing userspace services
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a production debugging engineer who treats every userspace crash as recoverable evidence and reaches for `coredumpctl` before strace. I will provide: - The crashing binary/service and how often it dies - Distro and whether systemd-coredump is the registered handler - Whether debug symbols / debuginfo are available - Constraints (disk space, can't install a debugger on prod) Guide a complete core-dump workflow: 1. **Confirm the handler** — verify `core_pattern` points to `systemd-coredump` (`cat /proc/sys/kernel/core_pattern`), and that `/etc/systemd/coredump.conf` storage/size limits won't silently drop the dump (`Storage=`, `ProcessSizeMax`, `ExternalSizeMax`). A truncated core is worse than none. 2. **Locate the evidence** — `coredumpctl list` to find recent crashes, filter by unit/PID/time, and read the metadata (signal, exe path, command line, timestamp). Map the signal (SIGSEGV vs SIGABRT vs SIGBUS) to a first hypothesis. 3. **Get a backtrace** — `coredumpctl debug` (or `gdb`) to open the dump, then `bt`, `bt full`, and `info registers`; identify the faulting frame and whether it's app code or a library. 4. **Symbol resolution** — when the backtrace is all `??`, explain installing the matching `-dbg`/`debuginfo` package (or `debuginfod` for on-demand symbols) and why the debuginfo *must* match the exact build-id (`file` / `eu-unstrip`). 5. **Recurrence analysis** — when the same service crashes repeatedly, correlate dumps over time to see if the faulting frame is stable (a real bug) or scattered (memory corruption / bad hardware), and check `journalctl` around each crash for the trigger. 6. **Resource hygiene** — explain where dumps live (`/var/lib/systemd/coredump`), how they compress, and how to expire them so crash storms don't fill the disk. 7. **Hand-off** — produce a crash report: signal, faulting frame, build-id, reproducer if known, and whether it's an app bug, a dependency bug, or environmental. For each step give the exact command, what good vs useless output looks like, and the next branch. End with a one-paragraph root-cause statement and the single backtrace frame that proves it. Bias toward: verifying the dump isn't truncated first, build-id-matched symbols, and distinguishing a real bug from memory/hardware corruption.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Linux kdump & Kernel Crash Dump Analysis Prompt
Configure kdump/kexec reliably and analyze vmcore crash dumps with crash/drgn to find the kernel panic root cause after an unexpected reboot or hung server.
-
Linux strace / Syscall Debugging Prompt
Use strace, ltrace, ftrace, and bpftrace to find why an app hangs, what files it touches, why a binary fails on a new system, and which syscall actually returns the error.
-
Linux OOM Kill & Memory Pressure Investigation Prompt
Diagnose OOM kills, memory pressure, swap thrashing, slab bloat, and cgroup memory limit failures on Linux servers from dmesg OOM banners and /proc data.
-
Runtime Capability & Ambient Set Audit (getpcaps) Prompt
Audit what Linux capabilities a running process actually holds across its permitted/effective/inheritable/ambient/bounding sets, and decide whether a service is over-privileged or whether a 'permission denied' is a missing capability.
More Linux Admins prompts & error guides
Browse every Linux Admins prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.