🐧 Analyze a systemd-coredump Core Dump with eu-unstrip and gdb

What systemd-coredump Does

On modern Linux distributions (Fedora, RHEL 8+, Ubuntu with systemd-coredump installed, Arch, openSUSE), the kernel’s core_pattern is wired to hand crashing processes to systemd-coredump instead of writing a core file in the working directory. When a process dies from a signal like SIGSEGV or SIGABRT, systemd captures the core, compresses it, records rich metadata in the journal, and stores the payload under /var/lib/systemd/coredump.

That is convenient for retention, but it changes the workflow. You no longer gdb ./myapp core. You go through coredumpctl, and when symbols are missing you lean on elfutils tools like eu-unstrip to figure out exactly which shared objects and build-ids were mapped into the crashed process.

Confirm the Core Pattern

Check that systemd-coredump is actually the handler:

cat /proc/sys/kernel/core_pattern

|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h

The leading pipe (|) means the kernel streams the core to that binary. If you instead see a plain path like core.%p, dumps land as files and systemd-coredump is not in play.

Listing and Inspecting Dumps

coredumpctl is the front door. List everything the system has captured:

coredumpctl list

TIME                        PID  UID  GID SIG     COREFILE EXE
Wed 2026-07-15 09:14:02 UTC 4821 1000 1000 SIGSEGV present  /usr/bin/myapp

Get the metadata for a specific dump, either by PID or by executable name:

coredumpctl info 4821

The info output shows the signal, the command line, the timestamp, the storage location, and a stack trace if debug symbols are already resolvable. When symbols are absent you will see a lot of ?? frames, which is the cue to identify the exact binaries and their build-ids.

Where the Core Is Stored

systemd-coredump keeps two things: structured metadata in the journal, and the compressed core payload on disk.

ls -lh /var/lib/systemd/coredump/

-rw-r-----. 1 root root 512K core.myapp.1000.<hash>.4821.<ts>.zst

Note the .zst (or .lz4) extension. You cannot feed a compressed core to gdb directly, which is why you extract it first. Retention and size limits live in /etc/systemd/coredump.conf (MaxUse, ProcessSizeMax, Storage).

Extracting the Raw Core

Pull an uncompressed core out to a normal file so the elfutils and gdb tooling can read it:

coredumpctl dump 4821 --output=/tmp/myapp.core

You now have a plain ELF core dump at /tmp/myapp.core. Everything below operates on that file.

Listing Loaded Modules with eu-unstrip

Before you chase symbols, you need the ground truth of what was actually mapped into the process and the exact build-id of each object. That is what eu-unstrip -n gives you:

eu-unstrip -n --core=/tmp/myapp.core

0x400000+0x21000 a3f1c9...b27 0x0 /usr/bin/myapp /usr/bin/myapp
0x7f2a...+0x1f0000 5d8e2f...a10 . /lib64/libc.so.6 libc.so.6
0x7f2b...+0x9000  9c14ab...ee0 . /lib64/libpthread.so.0 libpthread.so.0

Each row is: load address range, the 40-hex build-id, the debuginfo path (. means none separated, - means missing), the runtime file, and the module name. This is invaluable: the build-id is what uniquely ties a stripped binary to its matching debuginfo. If a build-id here does not match the debuginfo you have installed, gdb will silently give you wrong or missing symbols.

Step-by-Step Resolution

1. Verify the handler and locate the dump:

cat /proc/sys/kernel/core_pattern
coredumpctl list

2. Read the metadata and note the signal and executable:

coredumpctl info 4821

3. Extract the raw core for offline analysis:

coredumpctl dump 4821 --output=/tmp/myapp.core

4. Enumerate the mapped modules and their build-ids so you know exactly what to symbolize:

eu-unstrip -n --core=/tmp/myapp.core

5. Match debuginfo to each build-id. On Fedora/RHEL, install symbols for the crashing package and its libraries:

sudo dnf debuginfo-install myapp glibc

6. Get a fast backtrace without a full debugger using eu-stack, which resolves frames against the same build-ids:

eu-stack --core=/tmp/myapp.core

7. Open the dump in gdb for interactive analysis. The simplest path is to let coredumpctl launch gdb with the right executable already attached:

coredumpctl gdb 4821

Or point gdb at the extracted core yourself:

gdb /usr/bin/myapp /tmp/myapp.core

8. Inside gdb, confirm the frames now resolve to real function names and file:line locations:

(gdb) bt
#0  0x000055... in parse_config (path=0x0) at config.c:142
#1  0x000055... in main (argc=2, argv=...) at main.c:38

If frames still show ??, the build-id from eu-unstrip did not match the installed debuginfo. See the companion guide on missing debuginfo. Automating this triage across many hosts is a good fit for a repeatable prompt from our DevOps AI prompt library.

Prevention and Best Practices

• Keep systemd-coredump installed on production Linux hosts so crashes are captured with metadata instead of lost.
• Raise ProcessSizeMax in coredump.conf if large processes are being truncated, and watch MaxUse so /var/lib/systemd/coredump does not fill the disk.
• Record the build-id (eu-unstrip -n) alongside any bug report; it lets anyone fetch the exact matching debuginfo later.
• Prefer coredumpctl gdb over manual extraction when possible, since it selects the correct executable automatically.
• Ship debuginfo (or configure debuginfod) on at least one triage host so you can symbolize without rebuilding.

Frequently Asked Questions

Where does systemd-coredump store core dumps? The compressed payload lives under /var/lib/systemd/coredump/ (usually .zst or .lz4), and structured metadata is written to the systemd journal, queryable with coredumpctl.

How do I get an uncompressed core file for gdb? Run coredumpctl dump <PID> --output=/tmp/app.core. systemd stores dumps compressed, so you must extract before gdb or eu-stack can read them.

What does eu-unstrip -n —core do? It prints every module mapped into the crashed process with its load address and 40-character build-id, so you can match each stripped object to its exact debuginfo before symbolizing.

Do I have to use gdb, or is there something faster? eu-stack --core=<core> from elfutils produces a symbolized backtrace quickly without an interactive session, which is ideal for scripted triage. For more low-level debugging walkthroughs, see the Linux admin guides.