Managing Core Dumps with systemd-coredump
- WHAT?
From collecting and displaying core dumps, over providing stack traces for quick analysis to managing storage and retention, systemd-coredump enables you to investigate application crashes in a convenient way.
- WHY?
This article provides a complete overview of the tasks that can be performed with systemd-coredump, such as collecting and displaying core dumps, providing backtraces for quick analysis, and managing the storage of crash data.
- EFFORT
The average reading time of this article is approximately 40 minutes.
- GOAL
You will be able to analyze and manage application crashes using systemd-coredump.
- REQUIREMENTS
Basic understanding of Linux commands
Basic understanding of Linux processes
1 Using systemd-coredump to debug application crashes #
systemd-coredump collects and displays core dumps to analyze application crashes. A core dump is an image of the process memory at the time of termination.
1.1 Using systemd-coredump #
By default, when a process crashes, systemd-coredump performs the following actions:
Stores the core dump in
/var/lib/systemd/coredump/.Logs the event to the systemd journal.
Includes a backtrace where possible.
You can examine the dump file using coredumpctl or external tools such as gdb or crash.
Core dumps stored in /var/lib/systemd/coredump/ are by default automatically deleted after three days. This is configured through the d /var/lib/systemd/coredump
line in /usr/lib/tmpfiles.d/systemd.conf.
It is recommended not to edit /usr/lib/tmpfiles.d/systemd.conf directly, as the changes will be overwritten during the next system update. Hence, perform the following:
Copy the file to
/etc/tmpfiles.d/systemd.conf. As files in/etc/take precedence over those in/usr/lib/, your custom configuration is applied.Open the file and find the line:
d /var/lib/systemd/coredumpEdit the age parameter; for example, change
3dto7dto retain data for a week, or use theWnotation for longer periods.
2 Configuration of systemd-coredump #
systemd-coredump provides a centralized, automated framework for capturing and managing process crashes by intercepting kernel signals to store compressed memory images and metadata, which are then easily analyzed through the coredumpctl utility.
systemd-coredump is enabled by default. The primary configuration file is located at /etc/systemd/coredump.conf.
[Coredump] #Storage=external #Compress=yes #ProcessSizeMax=2G #ExternalSizeMax=2G #JournalSizeMax=767M #MaxUse= #KeepFree=
The following parameters in [Coredump] define how dumps are handled:
StorageDefines where to store dumps such as
none,external,journal. The default isexternal.For example:
[Coredump] # Directs dumps to be stored inside the system journal Storage=journal # Compress the data to save space within the journal Compress=yes # Limits the size of a single core dump stored in the journal to 500MB # If a process dump is larger than 500M, it will be dropped/ignored JournalSizeMax=500M # Processes larger than 1GB will not be handled at all ProcessSizeMax=1G
CompressEnables/disables compression for the stored core dump. By default, this is enabled.
ProcessSizeMaxThe maximum size of a core dump to be processed. The default size is
2G.ExternalSizeMaxThe maximum size for a dump stored on disk. The default size is
2G, and this also supports a value of infinity.JournalSizeMaxThe maximum size of core dumps that can be stored directly within the system journal (when Storage=journal is used). If a core dump exceeds this value, it will be ignored or handled according to other storage settings.
MaxUseThe maximum amount of disk space the core dump storage (typically
/var/lib/systemd/coredump/) is allowed to occupy. This value supports absolute sizes in bytes (e.g.,2G) or a percentage of the total filesystem size (e.g.,10%). Once this limit is reached, older core dumps are automatically deleted to make room for new ones.The size units are
B,K,M,G,T,P, andE.KeepFreeThe minimum amount of free disk space to maintain on the filesystem where core dumps are stored. This value can be specified in bytes (using suffixes such as K, M, G) or as a percentage of the total filesystem size (e.g.,
15%).systemd-coredumpwill stop saving new dumps if the available disk space would fall below this threshold.The size units are
B,K,M,G,T,P, andE.
A new instance of systemd-coredump is invoked for every crash. Configuration changes are applied immediately to the next crash; you do not need to restart any services.
2.1 Managing core dump storage #
To minimize the storage of sensitive information or prevent disk exhaustion in /var/lib/systemd/coredump, you can modify the Storage= option in /etc/systemd/coredump.conf.
If the Storage= field is omitted, set to an empty value, or set to an invalid string, systemd defaults to external.
The available storage options are:
Storage=nonethe event is logged to the journal, but the memory image (the core dump itself) is discarded and files are not created in/var/lib/systemd/coredump.Storage=journalThe core dump is compressed and stored directly within the systemd journal metadata and files are not created in/var/lib/systemd/coredump. The storage is managed by/etc/systemd/journald.conflimits.Storage=externalThe event is logged to the journal, and the memory image is saved as a separate file. The files are created in/var/lib/systemd/coredump.
As core dumps may not be preserved after a system restart (for example, if stored in volatile journal memory or a temporary directory), run the command below to save a core dump to a permanent file for later analysis:
coredumpctl -o my_app.dump dump >PID<
For more information, see man systemd-coredump, man coredumpctl, man core, and man coredump.conf for complete command and option listing.
3 Creating and analyzing a core dump #
In this section you force an application to quit without a proper shutdown. Thus the application creates a core dump that you can use to learn how to investigate core dumps.
The following procedure uses Vim to trigger a segmentation fault (segfault) for testing purposes.
Ensure the
debuginfo-poolanddebuginfo-updaterepositories are active for your registration and enabled.>sudozypper mr -e REPOSITORYInstall the debug information for vim.
>sudozypper install vim-debuginfoTo trigger a crash, open a file with Vim, then from another terminal, send a SIGSEGV signal to the process:
ps ax | grep vim # kill -s SIGSEGV <PID>
vim shows error messages, if any.
Vim: Caught deadly signal SEGV Vim: Finished. Segmentation fault (core dumped)
Verify if the crash was captured:
coredumpctl TIME PID UID GID SIG PRESENT EXE Wed 2019-11-12 11:56:47 PST 2345 1000 100 11 * /bin/vim
An asterisk
*in the PRESENT column indicates the dump file is available on disk.To view the stack trace and metadata of the latest crash:
coredumpctl info PID: 2345 (vim) UID: 0 (root) GID: 0 (root) Signal: 11 (SEGV) Timestamp: Wed 2019-11-12 11:58:05 PST Command Line: vim testfile Executable: /bin/vim Control Group: /user.slice/user-1000.slice/session-1.scope Unit: session-1.scope Slice: user-1000.slice Session: 1 Owner UID: 1000 (tux) Boot ID: b5c251b86ab34674a2222cef102c0c88 Machine ID: b43c44a64696799b985cafd95dc1b698 Hostname: linux-uoch Coredump: /var/lib/systemd/coredump/core.vim.0.b5c251b86ab34674a2222cef102 Message: Process 2345 (vim) of user 0 dumped core. Stack trace of thread 2345: #0 0x00007f21dd87e2a7 kill (libc.so.6) #1 0x000000000050cb35 may_core_dump (vim) #2 0x00007f21ddbfec70 __restore_rt (libpthread.so.0) #3 0x00007f21dd92ea33 __select (libc.so.6) #4 0x000000000050b4e3 RealWaitForChar (vim) #5 0x000000000050b86b mch_inchar (vim) [...]You can filter core dumps by PID, COMM (command), or EXE (full path to the executable).
To filter all core dumps for Vim:
coredumpctl info /bin/vim
To filter single core dump by PID
2345:coredumpctl info 2345
To load a specific crash for PID
2345directly into the debugger:coredumpctl gdb 2345
4 Legal Notice #
Copyright© 2006– 2026 SUSE LLC and contributors. All rights reserved.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”.
For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.
All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.