Info
Introduction
The first thing that you will do when opening a crash dump in WinDbg or WinDbg Preview is to run the !analyze -v command. Specifying the -v option provides the verbose output of the automated analysis that WinDbg performs on the crash dump. For the purposes of this tutorial I am going to use a mini-dump file that was created at the time of a BSOD event on an old Dell laptop. A copy of this dump file is attached below. The BSOD is triggered by the Windows operating system when it recognizes that an operation is about to cause serious problems in kernel mode and the operating system immediately shuts down, captures everything that was happening and puts it in the mini-dump file and/or the memory.dmp file if a full kernel memory dump has been requested. When opening a mini-dump file WinDbg will remind you - Mini Kernel Dump File: Only registers and stack trace are available. Not all WinDbg commands will work as a result of the limited information captured. The notes below will show you what to do to generate the output.
Info
Open the mini-dump file in WinDBG Preview, you will see the initial report that everything has loaded and the suggestion to click on the link that says !analyse -v. Do this and wait for the operation to complete.
The various sections of the output will now be explored in more detail.
Code:
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: fffff80fa011dfcc, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000008, value 0 = read operation, 1 = write operation
Arg4: fffff80fa011dfcc, address which referenced memory
This section tells you the
bugcheck code, some explanation of what it means and the four arguments that are recorded along with the bugcheck. Further information can be found by looking up the bugcheck code in the Microsoft documentation.
Bug Check 0xD1 DRIVER_IRQL_NOT_LESS_OR_EQUAL - Windows drivers
Info
You could also get this data from the .bugcheck command. When entering a command manually it is entered by typing into the area next to the 0: kd>
After the bugcheck code has been given the analysis continues with the debugging details. The initial results relate to the computer and you can get several useful items from this such as the operating system, the machine details including the BIOS version and date.
Code:
BUILD_VERSION_STRING: 16299.15.amd64fre.rs3_release.170928-1534
SYSTEM_MANUFACTURER: Dell Inc.
SYSTEM_PRODUCT_NAME: Latitude E6420
SYSTEM_VERSION: 01
BIOS_VENDOR: Dell Inc.
BIOS_VERSION: A24
BIOS_DATE: 05/12/2017
BASEBOARD_MANUFACTURER: Dell Inc.
BASEBOARD_PRODUCT: 032T9K
BASEBOARD_VERSION: A02
The next section just reiterates the bugcheck code and the four arguments listed now as P1 to P4. This is followed by an attempt to use the parameters and pinpoint where the fault has occurred. This does not always the provide the reason for the BSOD though - it can often be an event long before that actually initiated a series of commands that resulted in the OS recognising that a problem was being created and it needed to crash the system by issuing the bugcheck command (we will see this in a minute: nt!KeBugCheckEx).
Code:
BUGCHECK_P1: fffff80fa011dfcc
BUGCHECK_P2: 2
BUGCHECK_P3: 8
BUGCHECK_P4: fffff80fa011dfcc
READ_ADDRESS: fffff8030fc0d380: Unable to get MiVisibleState
Unable to get NonPagedPoolStart
Unable to get NonPagedPoolEnd
Unable to get PagedPoolStart
Unable to get PagedPoolEnd
fffff80fa011dfcc
CURRENT_IRQL: 2
FAULTING_IP:
ndis!ndisMiniportPMParametersUpdated+0
fffff80f`a011dfcc 4883ec28 sub rsp,28h
IP_IN_PAGED_CODE:
ndis!ndisMiniportPMParametersUpdated+0
fffff80f`a011dfcc 4883ec28 sub rsp,28h
Here the analysis has detected that a driver was trying to update the network miniport parameters and probably this was at too high an IRQL level. The current level for the CPU is shown as 2. Ndis.sys is a windows driver so it is not going to be the cause of the problem.
The next section is a report on the CPU of the device. The items are self explanatory.
Code:
CPU_COUNT: 4
CPU_MHZ: 9be
CPU_VENDOR: GenuineIntel
CPU_FAMILY: 6
CPU_MODEL: 2a
CPU_STEPPING: 7
CPU_MICROCODE: 6,2a,7,0 (F,M,S,R) SIG: 29'00000000 (cache) 29'00000000 (init)
Info
If a result is in hex code you can use the .formats command to translate it. See below where I have revealed that the CPU-MHz value is 2494 MHz:
Code:
0: kd> .formats 9be
Evaluate expression:
Hex: 00000000`000009be
Decimal: 2494
Octal: 0000000000000000004676
Binary: 00000000 00000000 00000000 00000000 00000000 00000000 00001001 10111110
Chars: ........
Time: Wed Dec 31 16:41:34 1969
Float: low 3.49484e-042 high 0
Double: 1.2322e-320
The next section of the !analyse output has some commands that can be used for blackboxbsd and blackboxpnp. These are relatively recent commands and I will not go into them now.
Following this there is a record of the DEFAULT_BUCKET_ID which can be useful to confirm that it was a normal crash rather than a Driver Verifier induced crash. The process that was running at the time of the crash is also recorded with the PROCESS_NAME label. This is well worth noting and if you are looking at a number of crash dumps it is one of the patterns to observe. If the same process is always involved it can be significant. This item is followed by some details of the computer name that is running the analysis - in this case my PC - with the time I ran the analysis and the version of WinDbg I was using.
Code:
DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT
BUGCHECK_STR: AV
PROCESS_NAME: dwm.exe
ANALYSIS_SESSION_HOST: DESKTOP-PH66QL2
ANALYSIS_SESSION_TIME: 04-01-2019 18:50:53.0479
ANALYSIS_VERSION: 10.0.18317.1001 amd64fre
The next section is not always present but in this crash a trapframe has been captured at the point of the page fault. The details of this trapframe are revealed and here you can see the command that was issued that caused the bugcheck to be initiated. It matches with the earlier FAULTING_IP result and is recorded again as the FAILED_INSTRUCTION_ADDRESS. Note that this address matches parameter P4.
Code:
TRAP_FRAME: fffff80311a64550 -- (.trap 0xfffff80311a64550)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000000000002 rbx=0000000000000000 rcx=ffffb502592a61a0
rdx=00000000ffffff02 rsi=0000000000000000 rdi=0000000000000000
rip=fffff80fa011dfcc rsp=fffff80311a646e8 rbp=fffff80311a64789
r8=0000000000000000 r9=0000000000000000 r10=0000000000000000
r11=ffffb502592a61a0 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na pe nc
ndis!ndisMiniportPMParametersUpdated:
fffff80f`a011dfcc 4883ec28 sub rsp,28h
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff8030f998ae9 to fffff8030f986930
FAILED_INSTRUCTION_ADDRESS:
ndis!ndisMiniportPMParametersUpdated+0
fffff80f`a011dfcc 4883ec28 sub rsp,28h
The next section is a view of the STACK. This is the stack associated with the processor core that was running at the time of the crash. It is processor core 0. We can tell this by looking at the number on the command line. It is "0: kd>". If it was processor core number 3 it would appear as "3: kd>".
Code:
STACK_TEXT:
fffff803`11a64408 fffff803`0f998ae9 : 00000000`0000000a fffff80f`a011dfcc 00000000`00000002 00000000`00000008 : nt!KeBugCheckEx
fffff803`11a64410 fffff803`0f9951fe : fffff788`c0001db8 ffffb502`5d077301 ffffe17c`557b2532 ffffb502`58f97628 : nt!KiBugCheckDispatch+0x69
fffff803`11a64550 fffff80f`a011dfcc : fffff80f`a00b8d25 ffffb502`59becb60 00000000`00000000 ffffb502`59becb60 : nt!KiPageFault+0x47e
fffff803`11a646e8 fffff80f`a00b8d25 : ffffb502`59becb60 00000000`00000000 ffffb502`59becb60 ffffb502`59a78000 : ndis!ndisMiniportPMParametersUpdated
fffff803`11a646f0 fffff80f`a00976d2 : 00000000`00000001 00000000`40030053 ffffb502`59a78788 00000000`00000000 : ndis!ndisMIndicatePMCapabilities+0xa1
fffff803`11a64720 fffff80f`a008098f : 00000000`00000001 00000000`00000002 ffff0001`00000001 fffff803`1220a582 : ndis!ndisMRawIndicateStatusEx+0x21c7e
fffff803`11a647f0 fffff80f`a008108d : ffffb502`59fd5000 ffffb502`59a78780 00000000`00000000 ffffb502`5a1c83c8 : ndis!ndisMpHookDefaultIndicateStatus+0xf
fffff803`11a64820 fffff803`12236d75 : ffff0001`00000001 00000000`00000070 ffffb502`590f0d01 00000000`00000000 : ndis!NdisMIndicateStatusEx+0x3d
fffff803`11a64860 fffff803`1221cad0 : ffff0001`00000000 ffffb502`59fd5000 ffffb502`59fd5000 ffffb502`590f0d30 : e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt+0x10d
fffff803`11a64960 fffff803`1221e701 : ffff0001`00000000 fffff80f`00000000 fffff803`11a64a20 ffffb502`59fd5000 : e1i63x64!INTERRUPT::IntInterruptDPC+0x2c4
fffff803`11a649d0 fffff803`1221ddb8 : ffffb502`580b4080 fffff803`00000000 fffff803`00000000 00000000`00000000 : e1i63x64!INTERRUPT::MsgIntMessageInterruptDPC+0x1ed
fffff803`11a64a60 fffff80f`a0088bd8 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`0000000f : e1i63x64!INTERRUPT::MiniportMessageInterruptDPC+0x28
fffff803`11a64aa0 fffff80f`a0088a0c : 00000000`00000000 00000000`00000000 ffffb502`59fbd338 ffffb502`5903f840 : ndis!ndisQueuedMiniportDpcWorkItem+0x198
fffff803`11a64bd0 fffff803`0f8d6f62 : fffff803`0e90df80 ffffb502`5903f840 fffff803`0e90b180 fffff803`0f918323 : ndis!ndisPeriodicReceivesTimer+0xdc
fffff803`11a64c70 fffff803`0f8d665f : 00000000`0000000e 00000000`00000000 000000b2`44ca0ec4 fffff803`0e90b180 : nt!KiExecuteAllDpcs+0x1d2
fffff803`11a64db0 fffff803`0f98de45 : 00000000`00000000 fffff803`0e90b180 ffffc680`b776bb80 fffff803`1013c550 : nt!KiRetireDpcList+0xdf
fffff803`11a64fb0 fffff803`0f98dc50 : fffffc2b`c4b0a800 fffff803`100e9356 ffffdfff`f3abd8b2 00000000`00000000 : nt!KxRetireDpcList+0x5
ffffc680`b776bac0 fffff803`0f98b4f5 : 00000000`00000000 fffff803`0f988981 0000020b`6a1c8d60 ffffb502`5bf5e080 : nt!KiDispatchInterruptContinue
ffffc680`b776baf0 fffff803`0f988981 : 0000020b`6a1c8d60 ffffb502`5bf5e080 00000000`0000a060 fffffc2b`c4b0a8c0 : nt!KiDpcInterruptBypass+0x25
ffffc680`b776bb00 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiInterruptDispatchNoLockNoEtw+0xb1
Info
You can use several commands to view the stack in different ways, here are some k; kL; kV. Do look them up in the help file for WinDbg for more details. The kV is my favourite because it reveals the trapframes as you can see below:
Code:
0: kd> kv
# Child-SP RetAddr : Args to Child : Call Site
00 fffff803`11a64408 fffff803`0f998ae9 : 00000000`0000000a fffff80f`a011dfcc 00000000`00000002 00000000`00000008 : nt!KeBugCheckEx
01 fffff803`11a64410 fffff803`0f9951fe : fffff788`c0001db8 ffffb502`5d077301 ffffe17c`557b2532 ffffb502`58f97628 : nt!KiBugCheckDispatch+0x69
02 fffff803`11a64550 fffff80f`a011dfcc : fffff80f`a00b8d25 ffffb502`59becb60 00000000`00000000 ffffb502`59becb60 : nt!KiPageFault+0x47e (TrapFrame @ fffff803`11a64550)
03 fffff803`11a646e8 fffff80f`a00b8d25 : ffffb502`59becb60 00000000`00000000 ffffb502`59becb60 ffffb502`59a78000 : ndis!ndisMiniportPMParametersUpdated
04 fffff803`11a646f0 fffff80f`a00976d2 : 00000000`00000001 00000000`40030053 ffffb502`59a78788 00000000`00000000 : ndis!ndisMIndicatePMCapabilities+0xa1
05 fffff803`11a64720 fffff80f`a008098f : 00000000`00000001 00000000`00000002 ffff0001`00000001 fffff803`1220a582 : ndis!ndisMRawIndicateStatusEx+0x21c7e
06 fffff803`11a647f0 fffff80f`a008108d : ffffb502`59fd5000 ffffb502`59a78780 00000000`00000000 ffffb502`5a1c83c8 : ndis!ndisMpHookDefaultIndicateStatus+0xf
07 fffff803`11a64820 fffff803`12236d75 : ffff0001`00000001 00000000`00000070 ffffb502`590f0d01 00000000`00000000 : ndis!NdisMIndicateStatusEx+0x3d
08 fffff803`11a64860 fffff803`1221cad0 : ffff0001`00000000 ffffb502`59fd5000 ffffb502`59fd5000 ffffb502`590f0d30 : e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt+0x10d
09 fffff803`11a64960 fffff803`1221e701 : ffff0001`00000000 fffff80f`00000000 fffff803`11a64a20 ffffb502`59fd5000 : e1i63x64!INTERRUPT::IntInterruptDPC+0x2c4
0a fffff803`11a649d0 fffff803`1221ddb8 : ffffb502`580b4080 fffff803`00000000 fffff803`00000000 00000000`00000000 : e1i63x64!INTERRUPT::MsgIntMessageInterruptDPC+0x1ed
0b fffff803`11a64a60 fffff80f`a0088bd8 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`0000000f : e1i63x64!INTERRUPT::MiniportMessageInterruptDPC+0x28
0c fffff803`11a64aa0 fffff80f`a0088a0c : 00000000`00000000 00000000`00000000 ffffb502`59fbd338 ffffb502`5903f840 : ndis!ndisQueuedMiniportDpcWorkItem+0x198
0d fffff803`11a64bd0 fffff803`0f8d6f62 : fffff803`0e90df80 ffffb502`5903f840 fffff803`0e90b180 fffff803`0f918323 : ndis!ndisPeriodicReceivesTimer+0xdc
0e fffff803`11a64c70 fffff803`0f8d665f : 00000000`0000000e 00000000`00000000 000000b2`44ca0ec4 fffff803`0e90b180 : nt!KiExecuteAllDpcs+0x1d2
0f fffff803`11a64db0 fffff803`0f98de45 : 00000000`00000000 fffff803`0e90b180 ffffc680`b776bb80 fffff803`1013c550 : nt!KiRetireDpcList+0xdf
10 fffff803`11a64fb0 fffff803`0f98dc50 : fffffc2b`c4b0a800 fffff803`100e9356 ffffdfff`f3abd8b2 00000000`00000000 : nt!KxRetireDpcList+0x5 (TrapFrame @ fffff803`11a64e70)
11 ffffc680`b776bac0 fffff803`0f98b4f5 : 00000000`00000000 fffff803`0f988981 0000020b`6a1c8d60 ffffb502`5bf5e080 : nt!KiDispatchInterruptContinue
12 ffffc680`b776baf0 fffff803`0f988981 : 0000020b`6a1c8d60 ffffb502`5bf5e080 00000000`0000a060 fffffc2b`c4b0a8c0 : nt!KiDpcInterruptBypass+0x25
13 ffffc680`b776bb00 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiInterruptDispatchNoLockNoEtw+0xb1 (TrapFrame @ ffffc680`b776bb00)
The stack is shown in reverse order of execution, the last call is at the top and earlier ones are below. I have highlighted an earlier call by the
e1i63x64.sys driver.
This is the first non-Microsoft driver we find in the stack reading down and it therefore becomes the one we most suspect as being the cause of the crash.
The final section of the output now goes into the failure cause. Here we get FOLLOW_UP items and these include the module with details of the driver version and date, the failure bucket ID, the Failure ID Hash string etc. You will need to use the .formats command to reveal the date for DEBUG_FLR_IMAGE_TIMESTAMP. You will find that there is a lot of repetition in the output at this stage, details of the operating system are given again and so I won't go over these a second time. I have highlighted those that are particularly important.
Code:
FOLLOWUP_IP:
e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt+10d
fffff803`12236d75 488b8c24e0000000 mov rcx,qword ptr [rsp+0E0h]
FAULT_INSTR_CODE: 248c8b48
SYMBOL_STACK_INDEX: 8
SYMBOL_NAME: e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt+10d
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: e1i63x64
IMAGE_NAME: e1i63x64.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 56da0235
IMAGE_VERSION: 12.15.22.6
STACK_COMMAND: .thread ; .cxr ; kb
BUCKET_ID_FUNC_OFFSET: 10d
FAILURE_BUCKET_ID: AV_CODE_AV_PAGED_IP_e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt
BUCKET_ID: AV_CODE_AV_PAGED_IP_e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt
PRIMARY_PROBLEM_CLASS: AV_CODE_AV_PAGED_IP_e1i63x64!INTERRUPT_E1Q_BASED::IntHandleMNGInterrupt
TARGET_TIME: 2018-05-04T18:51:26.000Z
OSBUILD: 16299
OSSERVICEPACK: 371
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 784
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt TerminalServer SingleUserTS Personal
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2018-03-30 04:22:49
BUILDDATESTAMP_STR: 170928-1534
BUILDLAB_STR: rs3_release
BUILDOSVER_STR: 10.0.16299.15.amd64fre.rs3_release.170928-1534
ANALYSIS_SESSION_ELAPSED_TIME: a19
ANALYSIS_SOURCE: KM
FAILURE_ID_HASH_STRING: km:av_code_av_paged_ip_e1i63x64!interrupt_e1q_based::inthandlemnginterrupt
FAILURE_ID_HASH: {11495abb-5259-76fb-9b5c-708572464cd7}
Followup: MachineOwner
That completes the run through of the output from the !analyse command in its verbose form. The conclusion is that a 3rd party driver is attempting to read or write to an invalid address. The solution in this case was to find an updated driver.
Further Reading:
Using the analyze Extension - Windows drivers
