R runuts Member Joined Jul 26, 2017 Posts 19 Location Cincinnati Aug 7, 2017 #21 x BlueRobot said: Could you please try and upload an additional dump file? I appear to be getting symbol errors with the current one? Click to expand... https://1drv.ms/f/s!AqVJ1YZxJQNi9FJevv6U8_Py2s1s
x BlueRobot said: Could you please try and upload an additional dump file? I appear to be getting symbol errors with the current one? Click to expand... https://1drv.ms/f/s!AqVJ1YZxJQNi9FJevv6U8_Py2s1s
x BlueRobot Administrator Staff member Joined May 7, 2013 Posts 10,195 Location %systemroot% Aug 8, 2017 #22 The call stack for the crash reveals the following: Code: 3: kd> knL # Child-SP RetAddr Call Site 00 ffffeb87`3bf01128 fffff802`b12267e9 nt!KeBugCheckEx 01 ffffeb87`3bf01130 fffff802`b1225f7c nt!KiBugCheckDispatch+0x69 02 ffffeb87`3bf01270 fffff802`b122171d nt!KiSystemServiceHandler+0x7c 03 ffffeb87`3bf012b0 fffff802`b11a5229 nt!RtlpExecuteHandlerForException+0xd 04 ffffeb87`3bf012e0 fffff802`b119cf1c nt!RtlDispatchException+0x3f9 05 ffffeb87`3bf019d0 fffff802`b12268ce nt!KiDispatchException+0x14c 06 ffffeb87`3bf02080 fffff802`b1224b57 nt!KiExceptionDispatch+0xce 07 ffffeb87`3bf02260 fffff808`f43004ea nt!KiPageFault+0x217 08 ffffeb87`3bf023f0 fffff808`f43001d4 NTFS!NtfsCommonQueryInformation+0x18a << We crashed here! 09 ffffeb87`3bf024f0 fffff808`f43000e0 NTFS!NtfsFsdDispatchSwitch+0xd4 0a ffffeb87`3bf02570 fffff802`b115799a NTFS!NtfsFsdDispatchWait+0x40 0b ffffeb87`3bf027e0 fffff802`b1838d5d nt!IopfCallDriver+0x56 0c ffffeb87`3bf02820 fffff802`b12428cb nt!IovCallDriver+0x275 0d ffffeb87`3bf02860 fffff808`f37285a3 nt!IofCallDriver+0x163b9b 0e ffffeb87`3bf028a0 fffff808`f3726be6 FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x1a3 0f ffffeb87`3bf02910 fffff802`b115799a FLTMGR!FltpDispatch+0xb6 10 ffffeb87`3bf02970 fffff802`b1838d5d nt!IopfCallDriver+0x56 11 ffffeb87`3bf029b0 fffff802`b12428cb nt!IovCallDriver+0x275 12 ffffeb87`3bf029f0 fffff802`b10dd21c nt!IofCallDriver+0x163b9b 13 ffffeb87`3bf02a30 fffff802`b14f0656 nt!IopCallDriverReference+0xdc 14 ffffeb87`3bf02aa0 fffff802`b1226353 nt!NtQueryInformationFile+0x586 15 ffffeb87`3bf02bd0 00007ff8`c609d234 nt!KiSystemServiceCopyEnd+0x13 16 000000eb`28979378 00000000`00000000 0x00007ff8`c609d234 The context record reveals that a invalid memory address was being referenced in the r8 register. Code: 3: kd> .cxr ffffeb873bf01a00 rax=0000000000000841 rbx=ffffad0684236bf0 rcx=0000000000000000 rdx=0000000000000003 rsi=ffffeb873bf02590 rdi=ffffad0684236e48 rip=fffff808f43004ea rsp=ffffeb873bf023f0 rbp=0000000000000000 r8=00000000ffff6ab0 r9=ffffd60ddd334180 r10=0000000000000000 r11=ffffeb873bf024e8 r12=0000000000000000 r13=ffffd60df030c520 r14=0000000000000005 r15=ffffd60df030c878 iopl=0 nv up ei ng nz na pe cy cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00010283 NTFS!NtfsCommonQueryInformation+0x18a: fffff808`f43004ea 4983b83001000000 cmp qword ptr [r8+130h],0 ds:002b:00000000`ffff6be0=???????????????? There isn't much information regarding how the r8 register was passed it's current value: Code: 3: kd> ub fffff808`f43004ea NTFS!NtfsCommonQueryInformation+0x16a: fffff808`f43004ca 0000 add byte ptr [rax],al fffff808`f43004cc 8b4708 mov eax,dword ptr [rdi+8] fffff808`f43004cf a820 test al,20h fffff808`f43004d1 0f855e040000 jne NTFS!NtfsCommonQueryInformation+0x5d5 (fffff808`f4300935) fffff808`f43004d7 4885ff test rdi,rdi fffff808`f43004da 0f84a3030000 je NTFS!NtfsCommonQueryInformation+0x523 (fffff808`f4300883) fffff808`f43004e0 4183fe30 cmp r14d,30h fffff808`f43004e4 0f8499030000 je NTFS!NtfsCommonQueryInformation+0x523 (fffff808`f4300883) Let's go back, and dump the call stack using the !stack extension, there were some interesting functions we should examine, most notably the IoCallDriver function. Code: 3: kd> !stack -p Call Stack : 15 frames ## Stack-Pointer Return-Address Call-Site 00 ffffeb873bf023f0 fffff808f43001d4 NTFS!NtfsCommonQueryInformation+18a Parameter[0] = ffffeb873bf02590 Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 01 ffffeb873bf024f0 fffff808f43000e0 NTFS!NtfsFsdDispatchSwitch+d4 Parameter[0] = ffffeb873bf02590 Parameter[1] = ffffd60df030c520 Parameter[2] = 0000000000000001 Parameter[3] = (unknown) 02 ffffeb873bf02570 fffff802b115799a NTFS!NtfsFsdDispatchWait+40 Parameter[0] = (unknown) Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 03 ffffeb873bf027e0 fffff802b1838d5d nt!IopfCallDriver+56 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 04 ffffeb873bf02820 fffff802b12428cb nt!IovCallDriver+275 Parameter[0] = ffffd60ddd334030 Parameter[1] = ffffd60df030c520 Parameter[2] = fffff808f37285a3 Parameter[3] = (unknown) 05 ffffeb873bf02860 fffff808f37285a3 nt!IofCallDriver+163b9b (perf) Parameter[0] = (unknown) Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 06 ffffeb873bf028a0 fffff808f3726be6 FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+1a3 Parameter[0] = ffffeb873bf02930 Parameter[1] = 0000000000000000 Parameter[2] = 0000000000000000 Parameter[3] = (unknown) 07 ffffeb873bf02910 fffff802b115799a FLTMGR!FltpDispatch+b6 Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 08 ffffeb873bf02970 fffff802b1838d5d nt!IopfCallDriver+56 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 09 ffffeb873bf029b0 fffff802b12428cb nt!IovCallDriver+275 Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = fffff802b10dd21c Parameter[3] = (unknown) 0a ffffeb873bf029f0 fffff802b10dd21c nt!IofCallDriver+163b9b (perf) Parameter[0] = ffffd60ddcf96ca0 << The device object Parameter[1] = ffffd60df030c520 << The IRP associated with the driver Parameter[2] = (unknown) Parameter[3] = (unknown) 0b ffffeb873bf02a30 fffff802b14f0656 nt!IopCallDriverReference+dc (perf) Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = 0000000000000001 Parameter[3] = ffffd60de01fe930 0c ffffeb873bf02aa0 fffff802b1226353 nt!NtQueryInformationFile+586 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 0d ffffeb873bf02bd0 00007ff8c609d234 nt!KiSystemServiceCopyEnd+13 Parameter[0] = 000001f55187bea8 Parameter[1] = 000001f539b75f00 Parameter[2] = 0000000000020000 Parameter[3] = 0000000000008000 It appears that an IRP was being sent to the file system, however, I wasn't able to dump the IRP which was associated with the device object. Code: 3: kd> !devobj ffffd60ddcf96ca0 Device object (ffffd60ddcf96ca0) is for: \FileSystem\FltMgr DriverObject ffffd60ddc5fa9f0 Current Irp 00000000 RefCount 0 Type 00000008 Flags 08040000 Dacl ffffae076868f9c0 DevExt ffffd60ddcf96df0 DevObjExt ffffd60ddcf96e48 ExtensionFlags (0x00000800) DOE_DEFAULT_SD_PRESENT Characteristics (0000000000) AttachedTo (Lower) ffffd60ddd334030 \FileSystem\NTFS Device queue is not busy. The device stack shows the following: Code: 3: kd> !devstack ffffd60ddcf96ca0 !DevObj !DrvObj !DevExt ObjectName > ffffd60ddcf96ca0 \FileSystem\FltMgr ffffd60ddcf96df0 ffffd60ddd334030 \FileSystem\NTFS ffffd60ddd334180 InfoMask field not found for _OBJECT_HEADER at ffffd60ddd334000 The same IRP is also present within the function which crashed too!
The call stack for the crash reveals the following: Code: 3: kd> knL # Child-SP RetAddr Call Site 00 ffffeb87`3bf01128 fffff802`b12267e9 nt!KeBugCheckEx 01 ffffeb87`3bf01130 fffff802`b1225f7c nt!KiBugCheckDispatch+0x69 02 ffffeb87`3bf01270 fffff802`b122171d nt!KiSystemServiceHandler+0x7c 03 ffffeb87`3bf012b0 fffff802`b11a5229 nt!RtlpExecuteHandlerForException+0xd 04 ffffeb87`3bf012e0 fffff802`b119cf1c nt!RtlDispatchException+0x3f9 05 ffffeb87`3bf019d0 fffff802`b12268ce nt!KiDispatchException+0x14c 06 ffffeb87`3bf02080 fffff802`b1224b57 nt!KiExceptionDispatch+0xce 07 ffffeb87`3bf02260 fffff808`f43004ea nt!KiPageFault+0x217 08 ffffeb87`3bf023f0 fffff808`f43001d4 NTFS!NtfsCommonQueryInformation+0x18a << We crashed here! 09 ffffeb87`3bf024f0 fffff808`f43000e0 NTFS!NtfsFsdDispatchSwitch+0xd4 0a ffffeb87`3bf02570 fffff802`b115799a NTFS!NtfsFsdDispatchWait+0x40 0b ffffeb87`3bf027e0 fffff802`b1838d5d nt!IopfCallDriver+0x56 0c ffffeb87`3bf02820 fffff802`b12428cb nt!IovCallDriver+0x275 0d ffffeb87`3bf02860 fffff808`f37285a3 nt!IofCallDriver+0x163b9b 0e ffffeb87`3bf028a0 fffff808`f3726be6 FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x1a3 0f ffffeb87`3bf02910 fffff802`b115799a FLTMGR!FltpDispatch+0xb6 10 ffffeb87`3bf02970 fffff802`b1838d5d nt!IopfCallDriver+0x56 11 ffffeb87`3bf029b0 fffff802`b12428cb nt!IovCallDriver+0x275 12 ffffeb87`3bf029f0 fffff802`b10dd21c nt!IofCallDriver+0x163b9b 13 ffffeb87`3bf02a30 fffff802`b14f0656 nt!IopCallDriverReference+0xdc 14 ffffeb87`3bf02aa0 fffff802`b1226353 nt!NtQueryInformationFile+0x586 15 ffffeb87`3bf02bd0 00007ff8`c609d234 nt!KiSystemServiceCopyEnd+0x13 16 000000eb`28979378 00000000`00000000 0x00007ff8`c609d234 The context record reveals that a invalid memory address was being referenced in the r8 register. Code: 3: kd> .cxr ffffeb873bf01a00 rax=0000000000000841 rbx=ffffad0684236bf0 rcx=0000000000000000 rdx=0000000000000003 rsi=ffffeb873bf02590 rdi=ffffad0684236e48 rip=fffff808f43004ea rsp=ffffeb873bf023f0 rbp=0000000000000000 r8=00000000ffff6ab0 r9=ffffd60ddd334180 r10=0000000000000000 r11=ffffeb873bf024e8 r12=0000000000000000 r13=ffffd60df030c520 r14=0000000000000005 r15=ffffd60df030c878 iopl=0 nv up ei ng nz na pe cy cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00010283 NTFS!NtfsCommonQueryInformation+0x18a: fffff808`f43004ea 4983b83001000000 cmp qword ptr [r8+130h],0 ds:002b:00000000`ffff6be0=???????????????? There isn't much information regarding how the r8 register was passed it's current value: Code: 3: kd> ub fffff808`f43004ea NTFS!NtfsCommonQueryInformation+0x16a: fffff808`f43004ca 0000 add byte ptr [rax],al fffff808`f43004cc 8b4708 mov eax,dword ptr [rdi+8] fffff808`f43004cf a820 test al,20h fffff808`f43004d1 0f855e040000 jne NTFS!NtfsCommonQueryInformation+0x5d5 (fffff808`f4300935) fffff808`f43004d7 4885ff test rdi,rdi fffff808`f43004da 0f84a3030000 je NTFS!NtfsCommonQueryInformation+0x523 (fffff808`f4300883) fffff808`f43004e0 4183fe30 cmp r14d,30h fffff808`f43004e4 0f8499030000 je NTFS!NtfsCommonQueryInformation+0x523 (fffff808`f4300883) Let's go back, and dump the call stack using the !stack extension, there were some interesting functions we should examine, most notably the IoCallDriver function. Code: 3: kd> !stack -p Call Stack : 15 frames ## Stack-Pointer Return-Address Call-Site 00 ffffeb873bf023f0 fffff808f43001d4 NTFS!NtfsCommonQueryInformation+18a Parameter[0] = ffffeb873bf02590 Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 01 ffffeb873bf024f0 fffff808f43000e0 NTFS!NtfsFsdDispatchSwitch+d4 Parameter[0] = ffffeb873bf02590 Parameter[1] = ffffd60df030c520 Parameter[2] = 0000000000000001 Parameter[3] = (unknown) 02 ffffeb873bf02570 fffff802b115799a NTFS!NtfsFsdDispatchWait+40 Parameter[0] = (unknown) Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 03 ffffeb873bf027e0 fffff802b1838d5d nt!IopfCallDriver+56 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 04 ffffeb873bf02820 fffff802b12428cb nt!IovCallDriver+275 Parameter[0] = ffffd60ddd334030 Parameter[1] = ffffd60df030c520 Parameter[2] = fffff808f37285a3 Parameter[3] = (unknown) 05 ffffeb873bf02860 fffff808f37285a3 nt!IofCallDriver+163b9b (perf) Parameter[0] = (unknown) Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 06 ffffeb873bf028a0 fffff808f3726be6 FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+1a3 Parameter[0] = ffffeb873bf02930 Parameter[1] = 0000000000000000 Parameter[2] = 0000000000000000 Parameter[3] = (unknown) 07 ffffeb873bf02910 fffff802b115799a FLTMGR!FltpDispatch+b6 Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = (unknown) Parameter[3] = (unknown) 08 ffffeb873bf02970 fffff802b1838d5d nt!IopfCallDriver+56 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 09 ffffeb873bf029b0 fffff802b12428cb nt!IovCallDriver+275 Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = fffff802b10dd21c Parameter[3] = (unknown) 0a ffffeb873bf029f0 fffff802b10dd21c nt!IofCallDriver+163b9b (perf) Parameter[0] = ffffd60ddcf96ca0 << The device object Parameter[1] = ffffd60df030c520 << The IRP associated with the driver Parameter[2] = (unknown) Parameter[3] = (unknown) 0b ffffeb873bf02a30 fffff802b14f0656 nt!IopCallDriverReference+dc (perf) Parameter[0] = ffffd60ddcf96ca0 Parameter[1] = ffffd60df030c520 Parameter[2] = 0000000000000001 Parameter[3] = ffffd60de01fe930 0c ffffeb873bf02aa0 fffff802b1226353 nt!NtQueryInformationFile+586 Parameter[0] = (unknown) Parameter[1] = (unknown) Parameter[2] = (unknown) Parameter[3] = (unknown) 0d ffffeb873bf02bd0 00007ff8c609d234 nt!KiSystemServiceCopyEnd+13 Parameter[0] = 000001f55187bea8 Parameter[1] = 000001f539b75f00 Parameter[2] = 0000000000020000 Parameter[3] = 0000000000008000 It appears that an IRP was being sent to the file system, however, I wasn't able to dump the IRP which was associated with the device object. Code: 3: kd> !devobj ffffd60ddcf96ca0 Device object (ffffd60ddcf96ca0) is for: \FileSystem\FltMgr DriverObject ffffd60ddc5fa9f0 Current Irp 00000000 RefCount 0 Type 00000008 Flags 08040000 Dacl ffffae076868f9c0 DevExt ffffd60ddcf96df0 DevObjExt ffffd60ddcf96e48 ExtensionFlags (0x00000800) DOE_DEFAULT_SD_PRESENT Characteristics (0000000000) AttachedTo (Lower) ffffd60ddd334030 \FileSystem\NTFS Device queue is not busy. The device stack shows the following: Code: 3: kd> !devstack ffffd60ddcf96ca0 !DevObj !DrvObj !DevExt ObjectName > ffffd60ddcf96ca0 \FileSystem\FltMgr ffffd60ddcf96df0 ffffd60ddd334030 \FileSystem\NTFS ffffd60ddd334180 InfoMask field not found for _OBJECT_HEADER at ffffd60ddd334000 The same IRP is also present within the function which crashed too!
x BlueRobot Administrator Staff member Joined May 7, 2013 Posts 10,195 Location %systemroot% Aug 8, 2017 #23 Code: BugCheck 109, {a39fe85d0428b72e, 0, 3fa3407c68e2b2f1, 101} Probably caused by : Unknown_Image ( ANALYSIS_INCONCLUSIVE ) It appears to be pool corruption again, which may help to explain the corrupted address within the r8 register. I would suggest running Driver Verifier until the system crashes again, hopefully you have a crash which indicates the issue. Code: 0: kd> !verifier Verify Flags Level 0x0002892b STANDARD FLAGS: [X] (0x00000000) Automatic Checks [X] (0x00000001) Special pool [X] (0x00000002) Force IRQL checking [X] (0x00000008) Pool tracking [ ] (0x00000010) I/O verification [X] (0x00000020) Deadlock detection [ ] (0x00000080) DMA checking [X] (0x00000100) Security checks [X] (0x00000800) Miscellaneous checks [X] (0x00020000) DDI compliance checking ADDITIONAL FLAGS: [ ] (0x00000004) Randomized low resources simulation [ ] (0x00000200) Force pending I/O requests [ ] (0x00000400) IRP logging [ ] (0x00002000) Invariant MDL checking for stack [ ] (0x00004000) Invariant MDL checking for driver [X] (0x00008000) Power framework delay fuzzing [ ] (0x00010000) Port/miniport interface checking [ ] (0x00040000) Systematic low resources simulation [ ] (0x00080000) DDI compliance checking (additional) [ ] (0x00200000) NDIS/WIFI verification [ ] (0x00800000) Kernel synchronization delay fuzzing [ ] (0x01000000) VM switch verification [ ] (0x02000000) Code integrity checks [X] Indicates flag is enabled It appears that still crash is consistently related to some form of memory corruption, and since we have carried out some hardware tests, leads me to believe that the issue is related to a driver still. Last edited: Aug 8, 2017
Code: BugCheck 109, {a39fe85d0428b72e, 0, 3fa3407c68e2b2f1, 101} Probably caused by : Unknown_Image ( ANALYSIS_INCONCLUSIVE ) It appears to be pool corruption again, which may help to explain the corrupted address within the r8 register. I would suggest running Driver Verifier until the system crashes again, hopefully you have a crash which indicates the issue. Code: 0: kd> !verifier Verify Flags Level 0x0002892b STANDARD FLAGS: [X] (0x00000000) Automatic Checks [X] (0x00000001) Special pool [X] (0x00000002) Force IRQL checking [X] (0x00000008) Pool tracking [ ] (0x00000010) I/O verification [X] (0x00000020) Deadlock detection [ ] (0x00000080) DMA checking [X] (0x00000100) Security checks [X] (0x00000800) Miscellaneous checks [X] (0x00020000) DDI compliance checking ADDITIONAL FLAGS: [ ] (0x00000004) Randomized low resources simulation [ ] (0x00000200) Force pending I/O requests [ ] (0x00000400) IRP logging [ ] (0x00002000) Invariant MDL checking for stack [ ] (0x00004000) Invariant MDL checking for driver [X] (0x00008000) Power framework delay fuzzing [ ] (0x00010000) Port/miniport interface checking [ ] (0x00040000) Systematic low resources simulation [ ] (0x00080000) DDI compliance checking (additional) [ ] (0x00200000) NDIS/WIFI verification [ ] (0x00800000) Kernel synchronization delay fuzzing [ ] (0x01000000) VM switch verification [ ] (0x02000000) Code integrity checks [X] Indicates flag is enabled It appears that still crash is consistently related to some form of memory corruption, and since we have carried out some hardware tests, leads me to believe that the issue is related to a driver still.
R runuts Member Joined Jul 26, 2017 Posts 19 Location Cincinnati Aug 11, 2017 #24 x BlueRobot said: Code: BugCheck 109, {a39fe85d0428b72e, 0, 3fa3407c68e2b2f1, 101} It appears that still crash is consistently related to some form of memory corruption, and since we have carried out some hardware tests, leads me to believe that the issue is related to a driver still. Click to expand... Code: I will set up driver verifier to run over the weekend. System has largely been stable the past several days. This may not have been the best solution, but I was able to download and burn to DVD a Windows Insider build. By booting with only basic drivers enabled, I was able to install and then update. With each update, the system stability seems to have improved. If the system will stay running over the weekend with driver verifier running, the problem may have been solved. Will report back on Monday.
x BlueRobot said: Code: BugCheck 109, {a39fe85d0428b72e, 0, 3fa3407c68e2b2f1, 101} It appears that still crash is consistently related to some form of memory corruption, and since we have carried out some hardware tests, leads me to believe that the issue is related to a driver still. Click to expand... Code: I will set up driver verifier to run over the weekend. System has largely been stable the past several days. This may not have been the best solution, but I was able to download and burn to DVD a Windows Insider build. By booting with only basic drivers enabled, I was able to install and then update. With each update, the system stability seems to have improved. If the system will stay running over the weekend with driver verifier running, the problem may have been solved. Will report back on Monday.