This is a draft document that was built and uploaded automatically. It may document beta software and be incomplete or even incorrect. Use this document at your own risk.

Jump to content
Capturing and Analyzing Kernel Crash Dumps with Kdump
SUSE Linux Enterprise Server for SAP applications 16.1

Capturing and Analyzing Kernel Crash Dumps with Kdump

Publication Date: 19 Nov 2026

When a Linux kernel panics, Kdump saves the memory of the crashed kernel to a file, vmcore, so that you can find out what caused the crash. Built on Kexec, Kdump keeps a second kernel ready in reserved memory and starts it the moment the main kernel crashes. This article explains how to set up Kdump on SUSE Linux Enterprise Server for SAP applications. You install and enable Kdump, reserve enough memory for it and test that it captures dumps. Finally, you analyze a captured dump.

Prerequisites

  • Enough free disk space at the dump target. By default, Kdump filters and compresses the dump, so it is smaller than the system memory.

1 What is Kdump and how does it use Kexec?

Kdump is a crash-dumping mechanism used to capture a system memory image at the moment a kernel panics or crashes, allowing administrators to perform debugging after a crash. It does not save the dump from the crashed kernel itself. Instead, it starts a second kernel via Kexec and captures the dump from there.

1.1 The dual-kernel model

To save the memory of a crashed kernel, Kdump relies on two kernels:

  • The production kernel is the kernel that normally runs your system and its workloads.

  • The capture kernel (also called the crash kernel or Kdump kernel) is a second, minimal kernel image with its own initramfs. It runs in an area of memory that is reserved for it. Kdump builds the initramfs so that it contains the drivers and tools, such as makedumpfile, needed to save the dump.

When the production kernel panics, it can no longer be trusted to write its own crash information to disk: Its data structures may be corrupted, and a normal reboot would overwrite the memory you are trying to inspect. Because the capture kernel runs from its own reserved memory region, the production kernel's memory contents remain intact and readable while the capture kernel is running.

1.2 How Kdump uses Kexec

Kexec is the mechanism that starts the capture kernel: a system call and the kexec command, which load a kernel into memory and start it directly, without going through the firmware (BIOS/UEFI) and the boot loader. Kdump is the service built around this mechanism. It reserves the memory for the capture kernel, builds its initramfs and defines how and where the dump is saved.

When the system boots and later crashes, the following happens:

  1. At boot, the kernel reserves an area of memory for the capture kernel. The size comes from the crashkernel= option on the kernel command line.

  2. kdump.service loads the capture kernel and its initramfs into the reserved area with kexec -p. The capture kernel stays idle.

  3. When the production kernel panics, it uses Kexec to start the capture kernel. The firmware and the boot loader do not run, so the memory of the crashed kernel stays intact.

  4. The capture kernel saves that memory to a vmcore file, together with the kernel log, and reboots the system.

Enabling Kdump does not change normal reboots. Kdump loads the capture kernel only for a crash, so a normal reboot still goes through the firmware and the boot loader. You can also use Kexec on its own to reboot faster, as described in the separate article “Using Kexec for Faster Reboots”.

1.3 The Kdump services and configuration file

Kdump runs as four systemd services. Enabling kdump.service also enables the other three. The services read their settings from /etc/sysconfig/kdump, where comments describe each option. For the frequently used options and how to apply changes, see Section 7, “Key options in /etc/sysconfig/kdump”.

kdump-early.service

Loads the capture kernel early during boot, before kdump.service runs.

kdump.service

Rebuilds the capture initramfs if necessary and loads the capture kernel into the reserved memory. The service fails if no memory is reserved.

kdump-commandline.service

Runs after kdump.service and checks whether the kernel command line contains the expected crashkernel= values. If they are missing or different, it updates the boot loader configuration, unless KDUMP_UPDATE_BOOTLOADER is set to false.

kdump-notify.service

Sends an e-mail notification about new crash dumps at boot, using mailx. It only does this if KDUMP_NOTIFICATION_TO contains an e-mail address and the dumps are saved locally.

1.4 What data does a captured vmcore file contain?

Once the capture kernel is running, it writes the production kernel's memory contents to a file called vmcore, together with a copy of the kernel's ring buffer log (dmesg) from the moment of the crash. Depending on the dump level set in KDUMP_DUMPLEVEL, a vmcore file can contain:

  • Kernel state: active kernel data structures, global variables and the call stack that was executing at the moment of the crash

  • Process information: the list of processes that were running, along with their individual stacks and registers

  • Memory pages: the actual data held in RAM by the kernel and by user-space applications

  • VMCOREINFO: a section that records how the kernel's memory was laid out, such as the kernel release, the page size and the addresses of kernel symbols, so that analysis tools can interpret the raw memory

You analyze this file after the system has rebooted, using tools such as crash or GDB. See Section 6, “Analyzing a captured vmcore file” for details.

2 Installing and enabling Kdump

Depending on how the system was installed, Kdump may not be installed or enabled. Make sure it is both, so that a crash produces a dump you can analyze afterward.

2.1 Installing Kdump

  1. Install the Kdump package. The package pulls in kexec-tools and makedumpfile as dependencies:

    > sudo  zypper install kdump
  2. Enable and start the Kdump service:

    > sudo  systemctl enable --now kdump

    If no crash kernel memory is reserved yet, kdump.service fails to start. The journal then shows No crashkernel memory reserved, Kdump cannot start. kdump-commandline.service still adds the crashkernel= values to the boot loader configuration. Kdump starts after the reboot in the next step.

  3. Reboot so the crash kernel memory reservation takes effect:

    > sudo  reboot

The default KDUMP_CRASHKERNEL="auto" reserves an automatically calculated amount of memory, which works for most systems. To check the value or set your own, see Section 3, “Sizing and reserving crash kernel memory”.

2.2 What happens when Kdump starts

When kdump.service starts, it rebuilds the capture initramfs if necessary and loads the capture kernel and the initramfs into the reserved memory area. Afterward, kdump-commandline.service checks whether the running kernel's command line (/proc/cmdline) contains the crashkernel= values that KDUMP_CRASHKERNEL specifies. With the default KDUMP_CRASHKERNEL="auto", these are the values that kdumptool calibrate calculates, and small differences in size are ignored. If the values are missing or different and KDUMP_UPDATE_BOOTLOADER is true (the default), kdump-commandline.service updates the boot loader configuration through pbl(8).

To check whether the service found and applied any change:

> sudo  journalctl -u kdump-commandline.service

Sample output from an AMD64/Intel 64 system that had no crash kernel memory reserved yet:

Oct 27 10:12:31 sles16 systemd[1]: Started Check and update kdump options on the kernel command line.
Oct 27 10:12:31 sles16 kdumptool[4136]: Kdump expects these crashkernel= values on the kernel command line:
Oct 27 10:12:31 sles16 kdumptool[4136]:     crashkernel=72M,low crashkernel=384M,high
Oct 27 10:12:31 sles16 kdumptool[4136]: (based on the output of 'kdumptool calibrate')
Oct 27 10:12:31 sles16 kdumptool[4136]: No crashkernel= found in /proc/cmdline. Kdump can't be started.
Oct 27 10:12:31 sles16 kdumptool[4136]: Updated the command line options in the bootloader.
Oct 27 10:12:31 sles16 kdumptool[4136]: Changes will take effect on next boot.
Oct 27 10:12:31 sles16 systemd[1]: kdump-commandline.service: Deactivated successfully.

If kdump-commandline.service updated the boot loader configuration, a reboot is required before the new crashkernel= values take effect and Kdump can start capturing dumps.

Important
Important: Keep manual crashkernel= values out of GRUB_CMDLINE_LINUX

On a system where KDUMP_UPDATE_BOOTLOADER is true, kdump-commandline.service replaces the crashkernel= values in GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub and never changes GRUB_CMDLINE_LINUX. A crashkernel= value in GRUB_CMDLINE_LINUX applies to every boot entry, including the recovery entries, which otherwise carry no reservation. On the normal boot entry, it then appears alongside the value the service sets, so the command line carries two conflicting reservations.

To set crashkernel= yourself, set KDUMP_UPDATE_BOOTLOADER="false" in /etc/sysconfig/kdump so the service no longer rewrites it, and put the value only in GRUB_CMDLINE_LINUX_DEFAULT.

3 Sizing and reserving crash kernel memory

Before Kdump can capture a crash dump, a dedicated area of memory must be reserved for the capture kernel. Use kdumptool calibrate to determine a suitable size, or set it manually if the automatic value is not sufficient.

3.1 Using the automatically calculated size

By default, KDUMP_CRASHKERNEL in /etc/sysconfig/kdump is set to auto. With this setting, kdumptool commandline derives the crash kernel memory size from kdumptool calibrate automatically. You do not need to calculate or set anything manually to get started. Run the calibration tool directly to see the values it would use:

Example 1: Determining the recommended crash kernel memory size
> sudo  kdumptool calibrate

Sample output from an AMD64/Intel 64 system:

Total: 4095
Low: 72
High: 384
MinLow: 72
MaxLow: 666
MinHigh: 0
MaxHigh: 2048
NonCMA: 300
CMA: 640

All values are given in megabytes. On AMD64/Intel 64, Low is the recommended reservation in the DMA32 zone (memory up to the 4 GiB mark), and High is the recommended reservation from the remaining available memory. On POWER, the whole reservation is reported as Low, and High is 0. MinLow/MaxLow and MinHigh/MaxHigh give the valid range around those recommendations. NonCMA and CMA are the suggested sizes for the two parts of the reservation when KDUMP_USE_CMA is enabled. See Section 7, “Key options in /etc/sysconfig/kdump”.

Note
Note: Calibrated values depend on the system

The exact numbers returned by kdumptool calibrate depend on the amount of RAM and the hardware configuration of the machine it runs on. Always run it on the target system itself rather than reusing values from another machine.

3.2 Crash kernel memory formats

When you set KDUMP_CRASHKERNEL manually, the value is either auto or one or more crashkernel= options in one of these formats:

Fixed size

crashkernel=SIZE[@OFFSET] reserves a fixed amount of memory; for example, crashkernel=256M. The optional offset is the physical address where the reserved area starts. A suitable size is the sum of the Low and High values from kdumptool calibrate. On POWER, auto produces this form.

Low and high split

crashkernel=SIZE_LOWM,low crashkernel=SIZE_HIGHM,high reserves one area below the 4 GiB mark and one from the remaining memory; for example, crashkernel=72M,low crashkernel=384M,high. The kernel accepts this form on AMD64/Intel 64 only. On AMD64/Intel 64, auto produces this form when kdumptool calibrate reports a Low value above 0. Otherwise, it sets only the ,high value.

Size depending on installed RAM

crashkernel=RANGE1:SIZE1[,RANGE2:SIZE2,...][@OFFSET] reserves a size that depends on the amount of installed RAM. Each range has the form START-[END], where the start is inclusive and the end is exclusive. For example, crashkernel=512M-2G:64M,2G-:128M reserves:

  • Nothing on systems with less than 512 MiB of RAM

  • 64 MiB on systems with 512 MiB up to but not including 2 GiB

  • 128 MiB on systems with 2 GiB or more

3.3 Setting a manual value

Override the automatically calculated reservation manually if it is not sufficient, for example, if Kdump still runs out of memory while capturing a dump. Take the value that kdumptool calibrate recommends and, if the system has multiple LUN kernel paths (paths to storage devices) attached, add roughly half the number of LUNs to it:

SIZE = RECOMMENDATION + (LUNS / 2)

Where LUNS is the maximum number of LUN kernel paths you expect the system to ever have (excluding multipath devices, which are ignored). To count the LUNs currently visible:

>  grep -c Lun /proc/scsi/scsi

On AMD64/Intel 64, use the High value as RECOMMENDATION. Set the result, together with the unmodified Low value as LOW, in /etc/sysconfig/kdump:

KDUMP_CRASHKERNEL="crashkernel=SIZEM,high crashkernel=LOWM,low"

When the reservation is placed above 4 GiB, Low is at least 72 MiB. Of these, 64 MiB are for software I/O TLB (swiotlb) bounce buffers and 8 MiB for overflow and other DMA buffers.

On POWER, only the Low value applies. Use it as RECOMMENDATION and set the result in /etc/sysconfig/kdump:

KDUMP_CRASHKERNEL="crashkernel=SIZEM"

3.4 Applying a changed reservation

After changing KDUMP_CRASHKERNEL, a full system reboot is required, since the crash kernel memory reservation is applied at boot time. See Section 2, “Installing and enabling Kdump” for how the new value is picked up.

4 Saving crash dumps to a remote target

By default, Kdump saves crash dumps locally under /var/crash. On systems with limited local storage, or where dumps must be collected centrally, configure KDUMP_SAVEDIR to save dumps to a remote target instead.

4.1 Supported remote storage protocols

KDUMP_SAVEDIR in /etc/sysconfig/kdump accepts a URL in the form PROTOCOL://SPECIFICATION. Install the client package matching the protocol you plan to use:

  • nfs-client, for nfs:// targets

  • cifs-utils, for cifs:// targets (Samba/Windows shares)

  • lftp, for ftp:// targets

  • openssh-clients, for ssh:// and sftp:// targets

For example, to save dumps to an FTP server:

KDUMP_SAVEDIR="ftp://USERNAME:PASSWORD@FTP_HOST:PORT/var/crash"

4.2 Saving dumps over SSH

A crash dump can contain sensitive data from system memory. SSH keeps that data encrypted in transit. SFTP is the preferred transport, since it does not require the target host to run any special server-side commands beyond the SFTP subsystem, which is enabled by default on SLES for SAP.

  1. Make the target host's identity known to Kdump ahead of time, so a dump is never sent to an impersonating host. Connect to the target once as root from the Kdump host so its key is recorded:

    > sudo  ssh TARGET_HOST
  2. Create a dedicated key for Kdump rather than reusing root's own key:

    > sudo  ssh-keygen -f /root/.ssh/kdump_key -N ""

    Leave the passphrase empty, since the capture kernel environment cannot prompt for one.

  3. Authorize the new key on the target host:

    > sudo  ssh-copy-id -i /root/.ssh/kdump_key TARGET_HOST
  4. In /etc/sysconfig/kdump, point Kdump at the new key and set the save directory:

    KDUMP_SSH_IDENTITY="/root/.ssh/kdump_key"
    KDUMP_SAVEDIR="sftp://TARGET_HOST/path/to/dumps"
  5. Restart Kdump to apply the new configuration:

    > sudo  systemctl restart kdump

5 Verifying that Kdump captures a crash dump

After configuring Kdump, validate the setup by triggering a test crash. Verifying dump capture in advance ensures that crash dumps (vmcore) are generated as expected during an unscheduled system failure.

Warning
Warning: This test crashes the system

Triggering a kernel panic immediately restarts the system, the same as a real crash would. Make sure no important workloads are running and no other users are logged in before you continue.

  1. Force an emergency sync of cached data to disk, to reduce the chance of data loss from the deliberate crash:

    >  echo s | sudo tee /proc/sysrq-trigger
  2. Force an emergency remount of file systems to read-only, to reduce the risk of corruption:

    >  echo u | sudo tee /proc/sysrq-trigger
  3. Trigger the crash:

    >  echo c | sudo tee /proc/sysrq-trigger

    The system panics immediately. Kdump captures a dump, then the system reboots on its own. Saving the dump can take a few minutes, depending on how much memory needs to be written.

  4. Once the system is back, confirm a dump was captured:

    Example 2: Confirming a captured crash dump
    > sudo  ls -la /var/crash/

    Sample output:

    drwx------  1 root root  42 Oct 27 10:13 2026-10-27-10-13

    Kdump creates one directory per captured dump, named after the date and time of the crash (YYYY-MM-DD-HH-MM). Inside it:

    Example 3: Contents of a captured dump directory
    > sudo  ls -la "/var/crash/DUMP_DIRECTORY/"

    Sample output:

    -rw-------  1 root root    81554 Oct 27 10:13 dmesg
    -rw-------  1 root root      308 Oct 27 10:13 README.txt
    -rw-------  1 root root 68839116 Oct 27 10:13 vmcore
    • dmesg is the kernel's own ring buffer log at the moment of the crash.

    • vmcore is the actual memory snapshot. See Section 6, “Analyzing a captured vmcore file” to read it.

    • README.txt summarizes the capture itself: kernel version, crash and dump time stamps, dump level and dump format. Check it first if a capture looks incomplete or was truncated.

6 Analyzing a captured vmcore file

Once Kdump has captured a vmcore file, use the crash utility to inspect it and find out what the kernel was doing at the moment it panicked.

6.1 Installing the analysis tools

  1. Enable the product's Debug repository. It is disabled by default. List your repositories to find its exact alias, then enable it:

    > sudo  zypper lr | grep -i debug
    > sudo  zypper modifyrepo --enable DEBUG_REPO_ALIAS
  2. Install crash and the debuginfo package of the running kernel. The package name contains the kernel flavor, the last part of the uname -r output. For the default flavor, this is kernel-default-debuginfo. The following command inserts the flavor automatically:

    > sudo  zypper install crash kernel-$(uname -r | sed 's/.*-//')-debuginfo
    Note
    Note: The debuginfo package version must match the running kernel exactly

    If the debuginfo package installs a version newer than the kernel you crashed (for example, because an update arrived after the crash but before you reboot), crash refuses to load it with a version mismatch error. Reboot onto the newest installed kernel first, and confirm uname -r matches the Kernel version line in the crash directory's README.txt before proceeding.

  3. The debuginfo package alone is not enough: On SLES for SAP, its vmlinux.debug file contains only debug symbols, not the actual kernel code and data crash needs alongside them. Place the real kernel image from the matching kernel package next to vmlinux.debug. On AMD64/Intel 64, the kernel package contains it compressed, so decompress it:

    >  xz -dc /usr/lib/modules/$(uname -r)/vmlinux.xz | \
        sudo tee /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux > /dev/null

    On POWER, the kernel package contains vmlinux uncompressed. Copy it instead:

    > sudo  cp /usr/lib/modules/$(uname -r)/vmlinux \
        /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux

6.2 Opening the vmcore file in crash

Start crash, giving it the kernel image you just placed and the captured vmcore file:

> sudo  crash /usr/lib/debug/usr/lib/modules/$(uname -r)/vmlinux \
    "/var/crash/DUMP_DIRECTORY/vmcore"

Sample output, from a crash deliberately triggered as described in Section 5, “Verifying that Kdump captures a crash dump”:

      KERNEL: /usr/lib/debug/usr/lib/modules/6.12.0-160100.7-default/vmlinux
   DEBUGINFO: /usr/lib/debug/usr/lib/modules/6.12.0-160100.7-default//vmlinux.debug
    DUMPFILE: /var/crash/2026-10-27-10-13/vmcore  [PARTIAL DUMP]
        CPUS: 2
        DATE: Tue Oct 27 10:13:42 CET 2026
      UPTIME: 00:00:21
LOAD AVERAGE: 0.59, 0.14, 0.05
       TASKS: 180
    NODENAME: sles16
     RELEASE: 6.12.0-160100.7-default
     VERSION: #1 SMP PREEMPT_DYNAMIC Wed Sep 23 08:40:15 UTC 2026 (3a3c8e4)
     MACHINE: x86_64  (2304 Mhz)
      MEMORY: 4 GB
       PANIC: "Kernel panic - not syncing: sysrq triggered crash"
         PID: 1290
     COMMAND: "tee" 1
        TASK: ffff8d67dd620000  [THREAD_INFO: ffff8d67dd620000]
         CPU: 1
       STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 1290     TASK: ffff8d67dd620000  CPU: 1    COMMAND: "tee"
 #0 [ffffce83c0d8f8e0] machine_kexec at ffffffffb26916a2
 #1 [ffffce83c0d8f940] __crash_kexec at ffffffffb27d2cab
 #2 [ffffce83c0d8fa00] panic at ffffffffb3360b1d
 #3 [ffffce83c0d8fa80] sysrq_handle_crash at ffffffffb2ec2dba
 #4 [ffffce83c0d8fa88] __handle_sysrq.cold at ffffffffb339e7a8
 #5 [ffffce83c0d8fab8] write_sysrq_trigger at ffffffffb2ec3959
 #6 [ffffce83c0d8fae0] proc_reg_write at ffffffffb2b1be67
 #7 [ffffce83c0d8faf8] vfs_write at ffffffffb2a6a45b
 #8 [ffffce83c0d8fb90] ksys_write at ffffffffb2a6aac7
 #9 [ffffce83c0d8fbc8] do_syscall_64 at ffffffffb33bea1f
#10 [ffffce83c0d8ff40] entry_SYSCALL_64_after_hwframe at ffffffffb340012f
    RIP: 00007faad43c8f04  RSP: 00007ffeba327138  RFLAGS: 00000202
    RAX: ffffffffffffffda  RBX: 0000000000000002  RCX: 00007faad43c8f04
    RDX: 0000000000000002  RSI: 00007ffeba3272b0  RDI: 0000000000000003
    RBP: 00007ffeba3272b0   R8: 0000000000000004   R9: 0000000000000001
    R10: 00000000000001b6  R11: 0000000000000202  R12: 0000000000000002
    R13: 0000555607ac12c0  R14: 00007faad44aaea0  R15: 0000000000000001
    ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b
crash> quit

1

The process that happened to be running when the crash was triggered, not necessarily a sign of a problem with that process. Here it is tee because the crash in this example was deliberately triggered with echo c | sudo tee /proc/sysrq-trigger. On a real, unplanned crash this would show whatever command was running at the time.

The banner shows the panic reason, the running process and basic system information. bt without arguments shows the call stack of the process active at the moment of the crash, here showing the path from the sysrq-trigger write down to machine_kexec, which is Kexec handing control to the capture kernel.

6.3 Useful follow-up commands

ps

Lists the processes that were running at the moment of the crash.

sym SYMBOL_OR_ADDRESS

Resolves a kernel symbol to its address, or an address back to a symbol.

files

Lists the open file descriptors of the current process.

kmem

Displays kernel memory usage details.

crash has extensive built-in help: Run help COMMAND inside a session for details on any of these.

7 Key options in /etc/sysconfig/kdump

The following table summarizes the /etc/sysconfig/kdump options you need most often.

Table 1: Frequently used /etc/sysconfig/kdump options
OptionPurpose
KDUMP_CRASHKERNELCrash kernel memory reservation. auto (default) uses the value from kdumptool calibrate. See Section 3, “Sizing and reserving crash kernel memory”.
KDUMP_UPDATE_BOOTLOADERWhen true (default), kdump-commandline.service updates the boot loader configuration whenever the running kernel command line does not match the expected crashkernel= options. Set to false to only check and warn, without modifying the boot loader.
KDUMP_USE_CMAExperimental (default: false). When true, the crash kernel reservation is split into a standard part and a part taken from the Contiguous Memory Allocator (CMA); for example, crashkernel=300M crashkernel=640M,cma. The CMA part stays available to user space on the running system. Kdump can become less reliable because DMA transfers set up by the production kernel can corrupt the memory of the capture kernel. KDUMP_DUMPLEVEL must filter out user-space pages, as the default value 31 does.
KDUMP_KERNELVERKernel version string to use for the capture kernel. Leave empty to auto-detect a suitable kernel.
KDUMP_CPUSNumber of CPUs available to the capture kernel environment (default: 4, or all available CPUs if set to 0). Lowering this reduces the memory the capture kernel needs. Updating the kdump package keeps a value that is already set in /etc/sysconfig/kdump, so an updated system can still use the previous default of 32.
KDUMP_SAVEDIRWhere captured dumps are stored. Defaults to /var/crash. See Section 4, “Saving crash dumps to a remote target” for remote targets.
KDUMP_KEEP_OLD_DUMPSNumber of old dumps to keep. Honored only for local save directories. The default, 0, never deletes old dumps. -1 deletes all old dumps, so that only the newly saved dump remains.
KDUMP_FREE_DISK_SIZEMinimum free space, in megabytes, that must remain at the save location after writing a dump. If less space remains, Kdump deletes the dump again. Applies only to local save directories. Default: 64.
KDUMP_DUMPLEVELHow much of memory to include in the dump, from 0 (largest, most complete dump) to 31 (smallest, the default). See man 5 kdump for the table of levels.
KDUMP_DUMPFORMATFormat of the saved dump. The default, compressed, produces small dumps that only crash can analyze. ELF produces larger dumps that GDB can also read. none saves only the kernel log. See man 5 kdump for the other formats.
KDUMP_IMMEDIATE_REBOOTWhen true (default), the system reboots automatically once the dump is saved.
KDUMP_CONTINUE_ON_ERRORWhen true (default), the capture kernel continues with the next action after an error while saving or deleting dumps. Set to false to open a shell at that point instead. See Section 8, “Troubleshooting”.
KDUMP_VERBOSEBit mask that sets the verbosity of the dump process (default: 0). See Section 8, “Troubleshooting”.
KDUMP_COMMANDLINEIf set, Kdump uses this command line for the capture kernel instead of building one. The command line that Kdump builds by default contains options the capture kernel needs, such as root=kdump, irqpoll, reset_devices and numa=off. To add options, use KDUMP_COMMANDLINE_APPEND instead.
KDUMP_COMMANDLINE_APPENDExtra options to append to the command line of the capture kernel. For example, an ip= option configures the network of the capture kernel manually. Kdump then skips its own network configuration.
KDUMP_SSH_IDENTITYPath to the SSH private key used when KDUMP_SAVEDIR points at an ssh:// or sftp:// target. See Section 4, “Saving crash dumps to a remote target”.
Important
Important: Restart Kdump after editing its configuration

After changing /etc/sysconfig/kdump, run systemctl restart kdump for the change to take effect. An exception is KDUMP_CRASHKERNEL: Since it changes the boot-time memory reservation, it requires a full system reboot, not just a service restart. See Section 2, “Installing and enabling Kdump”.

8 Troubleshooting

Kdump usually fails at one of three points: The crash kernel memory reservation is too small, the capture kernel does not start or the dump is not saved. Use the checks below to narrow down which stage is failing.

8.1 Crash kernel memory reservation is insufficient

If a crash produces error messages about low memory, or the Out Of Memory (OOM) killer being invoked inside the capture kernel, the reserved crash kernel memory is too small for this system. The reservation can be too small even after following Section 3, “Sizing and reserving crash kernel memory”, since the right amount depends on the actual workload running at the time of the crash.

  1. Check the currently reserved amount from the kernel command line:

    >  cat /proc/cmdline

    Add up every crashkernel= value shown (a plain value, or separate ,low/,high values) to get the current total reservation in MiB.

  2. Edit /etc/sysconfig/kdump and roughly double KDUMP_CRASHKERNEL from the current total:

    KDUMP_CRASHKERNEL="crashkernel=DOUBLED_SIZEM"
  3. Reboot to apply the new reservation, then try to reproduce the crash again. See Section 5, “Verifying that Kdump captures a crash dump”.

    > sudo  reboot
  4. Repeat with smaller increases until the dump completes reliably. A note taken between the last failing and the last working size narrows down a reasonable final value faster than repeatedly doubling.

8.2 Other common issues

The screen stays blank during the capture kernel boot

Configure a serial console so the capture kernel's boot messages are captured somewhere you can read them. A blank local console gives you nothing to debug from. Once you have output, increase logging by setting KDUMP_VERBOSE="11" in /etc/sysconfig/kdump. The value 11 removes the quiet option from the command line of the capture kernel. It also turns on debugging output for makedumpfile and for the kdump-save script, which saves the dump.

makedumpfile or network configuration errors during capture

Set KDUMP_CONTINUE_ON_ERROR="false" or KDUMP_IMMEDIATE_REBOOT="false". Instead of rebooting automatically, this drops you into an interactive shell inside the capture environment when an error occurs, or once makedumpfile finishes, so you can inspect the problem directly.

Dracut errors during the capture kernel boot

Add dracut's own debugging options to KDUMP_COMMANDLINE_APPEND in /etc/sysconfig/kdump; for example, KDUMP_COMMANDLINE_APPEND="rd.debug".

The capture initramfs does not generate correctly

Regenerate it with debugging enabled to see where it fails:

> sudo  mkdumprd -f -d

To inspect the contents of the generated image directly:

> sudo  lsinitrd --unpack /var/lib/kdump/initrd

9 For more information