Capturing and Analyzing Kernel Crash Dumps with Kdump
When a Linux kernel panics, Kdump saves the memory of the crashed kernel to a
file, vmcore, so that you can find out what caused the crash.
Built on Kexec, Kdump keeps a second kernel ready in reserved memory and starts it
the moment the main kernel crashes. This article explains how to set up Kdump on
SUSE Linux Enterprise Server for SAP applications. You install and enable Kdump, reserve enough memory for it and test
that it captures dumps. Finally, you analyze a captured dump.
Prerequisites
Enough free disk space at the dump target. By default, Kdump filters and compresses the dump, so it is smaller than the system memory.
1 What is Kdump and how does it use Kexec? #
Kdump is a crash-dumping mechanism used to capture a system memory image at the moment a kernel panics or crashes, allowing administrators to perform debugging after a crash. It does not save the dump from the crashed kernel itself. Instead, it starts a second kernel via Kexec and captures the dump from there.
1.1 The dual-kernel model #
To save the memory of a crashed kernel, Kdump relies on two kernels:
The production kernel is the kernel that normally runs your system and its workloads.
The capture kernel (also called the crash kernel or Kdump kernel) is a second, minimal kernel image with its own
initramfs. It runs in an area of memory that is reserved for it. Kdump builds theinitramfsso that it contains the drivers and tools, such asmakedumpfile, needed to save the dump.
When the production kernel panics, it can no longer be trusted to write its own crash information to disk: Its data structures may be corrupted, and a normal reboot would overwrite the memory you are trying to inspect. Because the capture kernel runs from its own reserved memory region, the production kernel's memory contents remain intact and readable while the capture kernel is running.
1.2 How Kdump uses Kexec #
Kexec is the mechanism that starts the capture kernel: a system call and the
kexec command, which load a kernel into memory and start it directly,
without going through the firmware (BIOS/UEFI) and the boot loader. Kdump is the
service built around this mechanism. It reserves the memory for the capture kernel,
builds its initramfs and defines how and where the dump is saved.
When the system boots and later crashes, the following happens:
At boot, the kernel reserves an area of memory for the capture kernel. The size comes from the
crashkernel=option on the kernel command line.kdump.serviceloads the capture kernel and itsinitramfsinto the reserved area withkexec -p. The capture kernel stays idle.When the production kernel panics, it uses Kexec to start the capture kernel. The firmware and the boot loader do not run, so the memory of the crashed kernel stays intact.
The capture kernel saves that memory to a
vmcorefile, together with the kernel log, and reboots the system.
Enabling Kdump does not change normal reboots. Kdump loads the capture kernel only for a crash, so a normal reboot still goes through the firmware and the boot loader. You can also use Kexec on its own to reboot faster, as described in the separate article “Using Kexec for Faster Reboots”.
1.3 The Kdump services and configuration file #
Kdump runs as four systemd services. Enabling kdump.service also
enables the other three. The services read their settings from
/etc/sysconfig/kdump, where comments describe each option. For the
frequently used options and how to apply changes, see
Section 7, “Key options in /etc/sysconfig/kdump”.
kdump-early.serviceLoads the capture kernel early during boot, before
kdump.serviceruns.kdump.serviceRebuilds the capture
initramfsif necessary and loads the capture kernel into the reserved memory. The service fails if no memory is reserved.kdump-commandline.serviceRuns after
kdump.serviceand checks whether the kernel command line contains the expectedcrashkernel=values. If they are missing or different, it updates the boot loader configuration, unlessKDUMP_UPDATE_BOOTLOADERis set tofalse.kdump-notify.serviceSends an e-mail notification about new crash dumps at boot, using
mailx. It only does this ifKDUMP_NOTIFICATION_TOcontains an e-mail address and the dumps are saved locally.
1.4 What data does a captured vmcore file contain? #
Once the capture kernel is running, it writes the production kernel's memory contents to
a file called vmcore, together with a copy of the kernel's ring
buffer log (dmesg) from the moment of the crash. Depending on the
dump level set in KDUMP_DUMPLEVEL, a vmcore file can contain:
Kernel state: active kernel data structures, global variables and the call stack that was executing at the moment of the crash
Process information: the list of processes that were running, along with their individual stacks and registers
Memory pages: the actual data held in RAM by the kernel and by user-space applications
VMCOREINFO: a section that records how the kernel's memory was laid out, such as the kernel release, the page size and the addresses of kernel symbols, so that analysis tools can interpret the raw memory
You analyze this file after the system has rebooted, using tools such as
crash or GDB. See Section 6, “Analyzing a captured vmcore file” for details.
2 Installing and enabling Kdump #
Depending on how the system was installed, Kdump may not be installed or enabled. Make sure it is both, so that a crash produces a dump you can analyze afterward.
2.1 Installing Kdump #
Install the Kdump package. The package pulls in kexec-tools and makedumpfile as dependencies:
>sudozypper install kdumpEnable and start the Kdump service:
>sudosystemctl enable --now kdumpIf no crash kernel memory is reserved yet,
kdump.servicefails to start. The journal then showsNo crashkernel memory reserved, Kdump cannot start.kdump-commandline.servicestill adds thecrashkernel=values to the boot loader configuration. Kdump starts after the reboot in the next step.Reboot so the crash kernel memory reservation takes effect:
>sudoreboot
The default KDUMP_CRASHKERNEL="auto" reserves an automatically
calculated amount of memory, which works for most systems. To check the value or set your
own, see Section 3, “Sizing and reserving crash kernel memory”.
2.2 What happens when Kdump starts #
When kdump.service starts, it rebuilds the capture
initramfs if necessary and loads the capture kernel and the
initramfs into the reserved memory area. Afterward,
kdump-commandline.service checks whether the running kernel's command
line (/proc/cmdline) contains the crashkernel=
values that KDUMP_CRASHKERNEL specifies. With the default
KDUMP_CRASHKERNEL="auto", these are the values that
kdumptool calibrate calculates, and small differences in size are
ignored. If the values are missing or different and
KDUMP_UPDATE_BOOTLOADER is true (the default),
kdump-commandline.service updates the boot loader configuration
through pbl(8).
To check whether the service found and applied any change:
>sudojournalctl -u kdump-commandline.service
Sample output from an AMD64/Intel 64 system that had no crash kernel memory reserved yet:
Oct 27 10:12:31 sles16 systemd[1]: Started Check and update kdump options on the kernel command line. Oct 27 10:12:31 sles16 kdumptool[4136]: Kdump expects these crashkernel= values on the kernel command line: Oct 27 10:12:31 sles16 kdumptool[4136]: crashkernel=72M,low crashkernel=384M,high Oct 27 10:12:31 sles16 kdumptool[4136]: (based on the output of 'kdumptool calibrate') Oct 27 10:12:31 sles16 kdumptool[4136]: No crashkernel= found in /proc/cmdline. Kdump can't be started. Oct 27 10:12:31 sles16 kdumptool[4136]: Updated the command line options in the bootloader. Oct 27 10:12:31 sles16 kdumptool[4136]: Changes will take effect on next boot. Oct 27 10:12:31 sles16 systemd[1]: kdump-commandline.service: Deactivated successfully.
If kdump-commandline.service updated the boot loader configuration, a
reboot is required before the new crashkernel= values take effect and
Kdump can start capturing dumps.
crashkernel= values out of GRUB_CMDLINE_LINUX
On a system where KDUMP_UPDATE_BOOTLOADER is
true, kdump-commandline.service replaces the
crashkernel= values in GRUB_CMDLINE_LINUX_DEFAULT
in /etc/default/grub and never changes
GRUB_CMDLINE_LINUX. A crashkernel= value in
GRUB_CMDLINE_LINUX applies to every boot entry, including the
recovery entries, which otherwise carry no reservation. On the normal boot entry, it then appears
alongside the value the service sets, so the command line carries two conflicting
reservations.
To set crashkernel= yourself, set
KDUMP_UPDATE_BOOTLOADER="false" in
/etc/sysconfig/kdump so the service no longer rewrites it, and put
the value only in GRUB_CMDLINE_LINUX_DEFAULT.
3 Sizing and reserving crash kernel memory #
Before Kdump can capture a crash dump, a dedicated area of memory must be reserved
for the capture kernel. Use kdumptool calibrate to determine a
suitable size, or set it manually if the automatic value is not sufficient.
3.1 Using the automatically calculated size #
By default, KDUMP_CRASHKERNEL in
/etc/sysconfig/kdump is set to auto. With this
setting, kdumptool commandline derives the crash kernel memory size
from kdumptool calibrate automatically. You do not need to calculate or
set anything manually to get started. Run the calibration tool directly to see the values
it would use:
>sudokdumptool calibrate
Sample output from an AMD64/Intel 64 system:
Total: 4095 Low: 72 High: 384 MinLow: 72 MaxLow: 666 MinHigh: 0 MaxHigh: 2048 NonCMA: 300 CMA: 640
All values are given in megabytes. On AMD64/Intel 64,
Low is the recommended reservation in the DMA32 zone (memory up to
the 4 GiB mark), and High is the recommended reservation from
the remaining available memory. On POWER, the whole reservation is
reported as Low, and High is 0.
MinLow/MaxLow and
MinHigh/MaxHigh give the valid range around those
recommendations. NonCMA and CMA are the suggested
sizes for the two parts of the reservation when KDUMP_USE_CMA is
enabled. See Section 7, “Key options in /etc/sysconfig/kdump”.
The exact numbers returned by kdumptool calibrate depend on the
amount of RAM and the hardware configuration of the machine it runs on. Always run it
on the target system itself rather than reusing values from another machine.
3.2 Crash kernel memory formats #
When you set KDUMP_CRASHKERNEL manually, the value is either
auto or one or more crashkernel= options in one of
these formats:
- Fixed size
crashkernel=SIZE[@OFFSET]reserves a fixed amount of memory; for example,crashkernel=256M. The optional offset is the physical address where the reserved area starts. A suitable size is the sum of theLowandHighvalues fromkdumptool calibrate. On POWER,autoproduces this form.- Low and high split
crashkernel=SIZE_LOWM,low crashkernel=SIZE_HIGHM,highreserves one area below the 4 GiB mark and one from the remaining memory; for example,crashkernel=72M,low crashkernel=384M,high. The kernel accepts this form on AMD64/Intel 64 only. On AMD64/Intel 64,autoproduces this form whenkdumptool calibratereports aLowvalue above 0. Otherwise, it sets only the,highvalue.- Size depending on installed RAM
crashkernel=RANGE1:SIZE1[,RANGE2:SIZE2,...][@OFFSET]reserves a size that depends on the amount of installed RAM. Each range has the form START-[END], where the start is inclusive and the end is exclusive. For example,crashkernel=512M-2G:64M,2G-:128Mreserves:Nothing on systems with less than 512 MiB of RAM
64 MiB on systems with 512 MiB up to but not including 2 GiB
128 MiB on systems with 2 GiB or more
3.3 Setting a manual value #
Override the automatically calculated reservation manually if it is not sufficient, for
example, if Kdump still runs out of memory while capturing a dump. Take the value that
kdumptool calibrate recommends and, if the system has multiple LUN
kernel paths (paths to storage devices) attached, add roughly half the number of LUNs to
it:
SIZE = RECOMMENDATION + (LUNS / 2)
Where LUNS is the maximum number of LUN kernel paths you expect the system to ever have (excluding multipath devices, which are ignored). To count the LUNs currently visible:
> grep -c Lun /proc/scsi/scsi
On AMD64/Intel 64, use the High value as
RECOMMENDATION. Set the result, together with the unmodified
Low value as LOW, in
/etc/sysconfig/kdump:
KDUMP_CRASHKERNEL="crashkernel=SIZEM,high crashkernel=LOWM,low"
When the reservation is placed above 4 GiB, Low is at least
72 MiB. Of these, 64 MiB are for software I/O TLB (swiotlb)
bounce buffers and 8 MiB for overflow and other DMA buffers.
On POWER, only the
Low value applies. Use it as RECOMMENDATION
and set the result in /etc/sysconfig/kdump:
KDUMP_CRASHKERNEL="crashkernel=SIZEM"
3.4 Applying a changed reservation #
After changing KDUMP_CRASHKERNEL, a full system reboot is required,
since the crash kernel memory reservation is applied at boot time. See
Section 2, “Installing and enabling Kdump” for how the new value is picked up.
4 Saving crash dumps to a remote target #
By default, Kdump saves crash dumps locally under /var/crash.
On systems with limited local storage, or where dumps must be collected centrally,
configure KDUMP_SAVEDIR to save dumps to a remote target instead.
4.1 Supported remote storage protocols #
KDUMP_SAVEDIR in /etc/sysconfig/kdump accepts a
URL in the form PROTOCOL://SPECIFICATION.
Install the client package matching the protocol you plan to use:
nfs-client, for
nfs://targetscifs-utils, for
cifs://targets (Samba/Windows shares)lftp, for
ftp://targetsopenssh-clients, for
ssh://andsftp://targets
For example, to save dumps to an FTP server:
KDUMP_SAVEDIR="ftp://USERNAME:PASSWORD@FTP_HOST:PORT/var/crash"
4.2 Saving dumps over SSH #
A crash dump can contain sensitive data from system memory. SSH keeps that data encrypted in transit. SFTP is the preferred transport, since it does not require the target host to run any special server-side commands beyond the SFTP subsystem, which is enabled by default on SLES for SAP.
Make the target host's identity known to Kdump ahead of time, so a dump is never sent to an impersonating host. Connect to the target once as
rootfrom the Kdump host so its key is recorded:>sudossh TARGET_HOSTCreate a dedicated key for Kdump rather than reusing
root's own key:>sudossh-keygen -f /root/.ssh/kdump_key -N ""Leave the passphrase empty, since the capture kernel environment cannot prompt for one.
Authorize the new key on the target host:
>sudossh-copy-id -i /root/.ssh/kdump_key TARGET_HOSTIn
/etc/sysconfig/kdump, point Kdump at the new key and set the save directory:KDUMP_SSH_IDENTITY="/root/.ssh/kdump_key" KDUMP_SAVEDIR="sftp://TARGET_HOST/path/to/dumps"
Restart Kdump to apply the new configuration:
>sudosystemctl restart kdump
5 Verifying that Kdump captures a crash dump #
After configuring Kdump, validate the setup by triggering a test crash. Verifying dump
capture in advance ensures that crash dumps (vmcore) are generated as expected during an
unscheduled system failure.
Triggering a kernel panic immediately restarts the system, the same as a real crash would. Make sure no important workloads are running and no other users are logged in before you continue.
Force an emergency sync of cached data to disk, to reduce the chance of data loss from the deliberate crash:
>echo s | sudo tee /proc/sysrq-triggerForce an emergency remount of file systems to read-only, to reduce the risk of corruption:
>echo u | sudo tee /proc/sysrq-triggerTrigger the crash:
>echo c | sudo tee /proc/sysrq-triggerThe system panics immediately. Kdump captures a dump, then the system reboots on its own. Saving the dump can take a few minutes, depending on how much memory needs to be written.
Once the system is back, confirm a dump was captured:
Example 2: Confirming a captured crash dump #>sudols -la /var/crash/Sample output:
drwx------ 1 root root 42 Oct 27 10:13 2026-10-27-10-13
Kdump creates one directory per captured dump, named after the date and time of the crash (
YYYY-MM-DD-HH-MM). Inside it:Example 3: Contents of a captured dump directory #>sudols -la "/var/crash/DUMP_DIRECTORY/"Sample output:
-rw------- 1 root root 81554 Oct 27 10:13 dmesg -rw------- 1 root root 308 Oct 27 10:13 README.txt -rw------- 1 root root 68839116 Oct 27 10:13 vmcore
dmesgis the kernel's own ring buffer log at the moment of the crash.vmcoreis the actual memory snapshot. See Section 6, “Analyzing a capturedvmcorefile” to read it.README.txtsummarizes the capture itself: kernel version, crash and dump time stamps, dump level and dump format. Check it first if a capture looks incomplete or was truncated.
6 Analyzing a captured vmcore file #
Once Kdump has captured a vmcore file, use the crash utility to inspect
it and find out what the kernel was doing at the moment it panicked.
6.1 Installing the analysis tools #
Enable the product's
Debugrepository. It is disabled by default. List your repositories to find its exact alias, then enable it:>sudozypper lr | grep -i debug>sudozypper modifyrepo --enable DEBUG_REPO_ALIASInstall
crashand the debuginfo package of the running kernel. The package name contains the kernel flavor, the last part of theuname -routput. For thedefaultflavor, this iskernel-default-debuginfo. The following command inserts the flavor automatically:>sudozypper install crash kernel-$(uname -r | sed 's/.*-//')-debuginfoNote: The debuginfo package version must match the running kernel exactlyIf the debuginfo package installs a version newer than the kernel you crashed (for example, because an update arrived after the crash but before you reboot),
crashrefuses to load it with a version mismatch error. Reboot onto the newest installed kernel first, and confirmuname -rmatches theKernel versionline in the crash directory'sREADME.txtbefore proceeding.The debuginfo package alone is not enough: On SLES for SAP, its
vmlinux.debugfile contains only debug symbols, not the actual kernel code and datacrashneeds alongside them. Place the real kernel image from the matching kernel package next tovmlinux.debug. On AMD64/Intel 64, the kernel package contains it compressed, so decompress it:>xz -dc /usr/lib/modules/$(uname -r)/vmlinux.xz | \ sudo tee /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux > /dev/nullOn POWER, the kernel package contains
vmlinuxuncompressed. Copy it instead:>sudocp /usr/lib/modules/$(uname -r)/vmlinux \ /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux
6.2 Opening the vmcore file in crash #
Start crash, giving it the kernel image you just placed and the
captured vmcore file:
>sudocrash /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux \ "/var/crash/DUMP_DIRECTORY/vmcore"
Sample output, from a crash deliberately triggered as described in Section 5, “Verifying that Kdump captures a crash dump”:
KERNEL: /usr/lib/debug/usr/lib/modules/6.12.0-160100.7-default/vmlinux
DEBUGINFO: /usr/lib/debug/usr/lib/modules/6.12.0-160100.7-default//vmlinux.debug
DUMPFILE: /var/crash/2026-10-27-10-13/vmcore [PARTIAL DUMP]
CPUS: 2
DATE: Tue Oct 27 10:13:42 CET 2026
UPTIME: 00:00:21
LOAD AVERAGE: 0.59, 0.14, 0.05
TASKS: 180
NODENAME: sles16
RELEASE: 6.12.0-160100.7-default
VERSION: #1 SMP PREEMPT_DYNAMIC Wed Sep 23 08:40:15 UTC 2026 (3a3c8e4)
MACHINE: x86_64 (2304 Mhz)
MEMORY: 4 GB
PANIC: "Kernel panic - not syncing: sysrq triggered crash"
PID: 1290
COMMAND: "tee" 1
TASK: ffff8d67dd620000 [THREAD_INFO: ffff8d67dd620000]
CPU: 1
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 1290 TASK: ffff8d67dd620000 CPU: 1 COMMAND: "tee"
#0 [ffffce83c0d8f8e0] machine_kexec at ffffffffb26916a2
#1 [ffffce83c0d8f940] __crash_kexec at ffffffffb27d2cab
#2 [ffffce83c0d8fa00] panic at ffffffffb3360b1d
#3 [ffffce83c0d8fa80] sysrq_handle_crash at ffffffffb2ec2dba
#4 [ffffce83c0d8fa88] __handle_sysrq.cold at ffffffffb339e7a8
#5 [ffffce83c0d8fab8] write_sysrq_trigger at ffffffffb2ec3959
#6 [ffffce83c0d8fae0] proc_reg_write at ffffffffb2b1be67
#7 [ffffce83c0d8faf8] vfs_write at ffffffffb2a6a45b
#8 [ffffce83c0d8fb90] ksys_write at ffffffffb2a6aac7
#9 [ffffce83c0d8fbc8] do_syscall_64 at ffffffffb33bea1f
#10 [ffffce83c0d8ff40] entry_SYSCALL_64_after_hwframe at ffffffffb340012f
RIP: 00007faad43c8f04 RSP: 00007ffeba327138 RFLAGS: 00000202
RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007faad43c8f04
RDX: 0000000000000002 RSI: 00007ffeba3272b0 RDI: 0000000000000003
RBP: 00007ffeba3272b0 R8: 0000000000000004 R9: 0000000000000001
R10: 00000000000001b6 R11: 0000000000000202 R12: 0000000000000002
R13: 0000555607ac12c0 R14: 00007faad44aaea0 R15: 0000000000000001
ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b
crash> quit
The process that happened to be running when the crash was triggered, not
necessarily a sign of a problem with that process. Here it is |
The banner shows the panic reason, the running process and basic system information. bt without arguments shows the call stack of the process
active at the moment of the crash, here showing the path from the
sysrq-trigger write down to machine_kexec, which is
Kexec handing control to the capture kernel.
6.3 Useful follow-up commands #
psLists the processes that were running at the moment of the crash.
symSYMBOL_OR_ADDRESSResolves a kernel symbol to its address, or an address back to a symbol.
filesLists the open file descriptors of the current process.
kmemDisplays kernel memory usage details.
crash has extensive built-in help: Run help
COMMAND inside a session for details on any of these.
7 Key options in /etc/sysconfig/kdump #
The following table summarizes the /etc/sysconfig/kdump options
you need most often.
/etc/sysconfig/kdump options #| Option | Purpose |
|---|---|
KDUMP_CRASHKERNEL | Crash kernel memory reservation. auto (default) uses the
value from kdumptool calibrate. See
Section 3, “Sizing and reserving crash kernel memory”. |
KDUMP_UPDATE_BOOTLOADER | When true (default), kdump-commandline.service
updates the boot loader configuration whenever the running kernel command line does
not match the expected crashkernel= options. Set to
false to only check and warn, without modifying the boot loader. |
KDUMP_USE_CMA | Experimental (default: false). When
true, the crash kernel reservation is split into a standard part
and a part taken from the Contiguous Memory Allocator (CMA); for example,
crashkernel=300M crashkernel=640M,cma. The CMA part stays
available to user space on the running system. Kdump can become less reliable
because DMA transfers set up by the production kernel can corrupt the memory of the
capture kernel. KDUMP_DUMPLEVEL must filter out user-space pages,
as the default value 31 does. |
KDUMP_KERNELVER | Kernel version string to use for the capture kernel. Leave empty to auto-detect a suitable kernel. |
KDUMP_CPUS | Number of CPUs available to the capture kernel environment (default: 4, or all
available CPUs if set to 0). Lowering this reduces the memory the capture kernel
needs. Updating the kdump package keeps a value that is already
set in /etc/sysconfig/kdump, so an updated system can still use
the previous default of 32. |
KDUMP_SAVEDIR | Where captured dumps are stored. Defaults to /var/crash.
See Section 4, “Saving crash dumps to a remote target” for remote targets. |
KDUMP_KEEP_OLD_DUMPS | Number of old dumps to keep. Honored only for local save directories. The default,
0, never deletes old dumps. -1 deletes all old
dumps, so that only the newly saved dump remains. |
KDUMP_FREE_DISK_SIZE | Minimum free space, in megabytes, that must remain at the save location after
writing a dump. If less space remains, Kdump deletes the dump again. Applies only
to local save directories. Default: 64. |
KDUMP_DUMPLEVEL | How much of memory to include in the dump, from 0 (largest,
most complete dump) to 31 (smallest, the default). See
man 5 kdump for the table of levels. |
KDUMP_DUMPFORMAT | Format of the saved dump. The default, compressed, produces small
dumps that only crash can analyze. ELF produces
larger dumps that GDB can also read. none saves only the kernel
log. See man 5 kdump for the other formats. |
KDUMP_IMMEDIATE_REBOOT | When true (default), the system reboots automatically once the
dump is saved. |
KDUMP_CONTINUE_ON_ERROR | When true (default), the capture kernel continues with the next
action after an error while saving or deleting dumps. Set to false
to open a shell at that point instead. See Section 8, “Troubleshooting”. |
KDUMP_VERBOSE | Bit mask that sets the verbosity of the dump process (default: 0). See
Section 8, “Troubleshooting”. |
KDUMP_COMMANDLINE | If set, Kdump uses this command line for the capture kernel instead of
building one. The command line that Kdump builds by default contains options the
capture kernel needs, such as root=kdump,
irqpoll, reset_devices and
numa=off. To add options, use
KDUMP_COMMANDLINE_APPEND instead. |
KDUMP_COMMANDLINE_APPEND | Extra options to append to the command line of the capture kernel. For example,
an ip= option configures the network of the capture kernel
manually. Kdump then skips its own network configuration. |
KDUMP_SSH_IDENTITY | Path to the SSH private key used when KDUMP_SAVEDIR points
at an ssh:// or sftp:// target. See
Section 4, “Saving crash dumps to a remote target”. |
After changing /etc/sysconfig/kdump, run
systemctl restart kdump for the change to take effect. An exception is
KDUMP_CRASHKERNEL: Since it changes the boot-time memory reservation, it
requires a full system reboot, not just a service restart. See Section 2, “Installing and enabling Kdump”.
8 Troubleshooting #
Kdump usually fails at one of three points: The crash kernel memory reservation is too small, the capture kernel does not start or the dump is not saved. Use the checks below to narrow down which stage is failing.
8.1 Crash kernel memory reservation is insufficient #
If a crash produces error messages about low memory, or the Out Of Memory (OOM) killer being invoked inside the capture kernel, the reserved crash kernel memory is too small for this system. The reservation can be too small even after following Section 3, “Sizing and reserving crash kernel memory”, since the right amount depends on the actual workload running at the time of the crash.
Check the currently reserved amount from the kernel command line:
>cat /proc/cmdlineAdd up every
crashkernel=value shown (a plain value, or separate,low/,highvalues) to get the current total reservation in MiB.Edit
/etc/sysconfig/kdumpand roughly doubleKDUMP_CRASHKERNELfrom the current total:KDUMP_CRASHKERNEL="crashkernel=DOUBLED_SIZEM"
Reboot to apply the new reservation, then try to reproduce the crash again. See Section 5, “Verifying that Kdump captures a crash dump”.
>sudorebootRepeat with smaller increases until the dump completes reliably. A note taken between the last failing and the last working size narrows down a reasonable final value faster than repeatedly doubling.
8.2 Other common issues #
- The screen stays blank during the capture kernel boot
Configure a serial console so the capture kernel's boot messages are captured somewhere you can read them. A blank local console gives you nothing to debug from. Once you have output, increase logging by setting
KDUMP_VERBOSE="11"in/etc/sysconfig/kdump. The value 11 removes thequietoption from the command line of the capture kernel. It also turns on debugging output formakedumpfileand for thekdump-savescript, which saves the dump.makedumpfileor network configuration errors during captureSet
KDUMP_CONTINUE_ON_ERROR="false"orKDUMP_IMMEDIATE_REBOOT="false". Instead of rebooting automatically, this drops you into an interactive shell inside the capture environment when an error occurs, or oncemakedumpfilefinishes, so you can inspect the problem directly.- Dracut errors during the capture kernel boot
Add dracut's own debugging options to
KDUMP_COMMANDLINE_APPENDin/etc/sysconfig/kdump; for example,KDUMP_COMMANDLINE_APPEND="rd.debug".- The capture
initramfsdoes not generate correctly Regenerate it with debugging enabled to see where it fails:
>sudomkdumprd -f -dTo inspect the contents of the generated image directly:
>sudolsinitrd --unpack /var/lib/kdump/initrd
9 For more information #
Man pages:
man 5 kdump,man 7 kdumpLinux kernel documentation on Kdump: https://docs.kernel.org/admin-guide/kdump/kdump.html
10 Legal Notice #
Copyright© 2006– 2026 SUSE LLC and contributors. All rights reserved.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”.
For SUSE trademarks, see https://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks.
All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors, nor the translators shall be held liable for possible errors or the consequences thereof.