[SOLVED] A whole range of BSoDs

niemiro · Jul 8, 2012

Hello all :)

Sorry for it being so long. An update. Basically, the BSODs continue. There were ~4 separate patterns. I have, after much effort, finally pinned each onto specific drivers, and am now left with one final pattern which I seem unable to solve. I have, however, really enjoyed working it, but would love a little bit more assistance.

I have basically tried all that you have suggested thus far, and I will re-read the thread, just to be sure. Hardware diagnostics came back good, although that was a little while ago, and I will re-run them, just to be sure.

However, I have a personal belief that this is a driver. The call stacks, memory address patterns, always an access violation, etc. etc. are absolutely identical in every dump. No variations whatsoever.

Most times it crashes without creating a minidump, and ~1 in 6 it creates a kernel memory dump.

I have two separate patterns 0x1E_C0000005, and 0x3B_C0000005. However, I strongly suspect that they have the same cause, once again due to huge similarities in the call stack and memory address patterns.

However, I have now noticed something even more odd, which you experts might tell me is not odd at all.

Code:

Microsoft (R) Windows Debugger Version 6.2.8229.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [D:\Users\Richard\Desktop\MEMORY (16).DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*D:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS Personal
Built by: 7601.17835.amd64fre.win7sp1_gdr.120503-2030
Machine Name:
Kernel base = 0xfffff800`0325b000 PsLoadedModuleList = 0xfffff800`0349f670
Debug session time: Sat Jul  7 19:29:27.427 2012 (UTC + 1:00)
System Uptime: 0 days 0:19:58.239
Loading Kernel Symbols
...............................................................
................................................................
..............................................
Loading User Symbols
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Loading unloaded module list
......
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck 1E, {ffffffffc0000005, fffff800032ff830, 0, ffffffffffffffff}
Probably caused by : memory_corruption ( nt!MiReplenishPageSlist+c0 )
Followup: MachineOwner
---------
4: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
KMODE_EXCEPTION_NOT_HANDLED (1e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: ffffffffc0000005, The exception code that was not handled
Arg2: fffff800032ff830, The address that the exception occurred at
Arg3: 0000000000000000, Parameter 0 of the exception
Arg4: ffffffffffffffff, Parameter 1 of the exception
Debugging Details:
------------------

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
FAULTING_IP: 
nt!MiReplenishPageSlist+c0
fffff800`032ff830 f00fba6b1000    lock bts dword ptr [rbx+10h],0
EXCEPTION_PARAMETER1:  0000000000000000
EXCEPTION_PARAMETER2:  ffffffffffffffff
READ_ADDRESS:  ffffffffffffffff 
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
BUGCHECK_STR:  0x1e_c0000005
DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT
PROCESS_NAME:  iexplore.exe
CURRENT_IRQL:  2
LAST_CONTROL_TRANSFER:  from fffff80003324d88 to fffff800032da1c0
CONTEXT:  480100161b0505f6 -- (.cxr 0x480100161b0505f6)
Unable to read context, NTSTATUS 0xC0000141
STACK_TEXT:  
fffff880`0c344dd8 fffff800`03324d88 : 00000000`0000001e ffffffff`c0000005 fffff800`032ff830 00000000`00000000 : nt!KeBugCheckEx
fffff880`0c344de0 fffff800`032d9842 : fffff880`0c3455b8 bffffa80`0888bf10 fffff880`0c345660 00000000`00000006 : nt! ?? ::FNODOBFM::`string'+0x48d3d
fffff880`0c345480 fffff800`032d814a : fffffa80`0fe7bbb0 fffff8a0`006a6650 fffffa80`0d9b1430 fffffa80`0deb4180 : nt!KiExceptionDispatch+0xc2
fffff880`0c345660 fffff800`032ff830 : ffffffff`ffffffff fffffa80`0f2dfb50 00000000`18030000 fffff800`03302106 : nt!KiGeneralProtectionFault+0x10a
fffff880`0c3457f0 fffff800`032fdfef : fffffa80`0cbfb338 00000000`0000007b fffffa80`0889c710 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
fffff880`0c345860 fffff800`032e7614 : 00000000`00000000 00000000`00000002 00000000`00000000 ffffffff`ffffffff : nt!MiRemoveAnyPage+0x24f
fffff880`0c345980 fffff800`032d82ee : 00000000`00000001 00000000`18037000 00000000`16822101 00000000`000000b0 : nt!MmAccessFault+0x1224
fffff880`0c345ae0 00000000`6ec06af7 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x16e
00000000`1f05d000 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x6ec06af7

FOLLOWUP_IP: 
nt!MiReplenishPageSlist+c0
fffff800`032ff830 f00fba6b1000    lock bts dword ptr [rbx+10h],0
SYMBOL_STACK_INDEX:  4
SYMBOL_NAME:  nt!MiReplenishPageSlist+c0
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: nt
DEBUG_FLR_IMAGE_TIMESTAMP:  4fa390f3
STACK_COMMAND:  .cxr 0x480100161b0505f6 ; kb
IMAGE_NAME:  memory_corruption
FAILURE_BUCKET_ID:  X64_0x1e_c0000005_VRF_nt!MiReplenishPageSlist+c0
BUCKET_ID:  X64_0x1e_c0000005_VRF_nt!MiReplenishPageSlist+c0
Followup: MachineOwner
---------
4: kd> !pfn fffff800`032ff830
    PFN 5554D5558854D85 at address FFFE7A8098FE8900
    flink       00000000  blink / share count 00000000  pteaddress 00000000
    reference count 0000    used entry count  0000      NonCached color 0   Priority 0
    restore pte 00000000  containing page        000000  Zeroed             
                   
4: kd> dt nt!_MMPFN fffff800`032ff830
   +0x000 u1               : <unnamed-tag>
   +0x008 u2               : <unnamed-tag>
   +0x010 PteAddress       : 0xfff5d8d3`850feb8b _MMPTE
   +0x010 VolatilePteAddress : 0xfff5d8d3`850feb8b Void
   +0x010 Lock             : 0n-2062554229
   +0x010 PteLong          : 0xfff5d8d3`850feb8b
   +0x018 u3               : <unnamed-tag>
   +0x01c UsedPageTableEntries : 0x20
   +0x01e VaType           : 0x1 ''
   +0x01f ViewCount        : 0xf ''
   +0x020 OriginalPte      : _MMPTE
   +0x020 AweReferenceCount : 0n-170333820
   +0x028 u4               : <unnamed-tag>
4: kd> !pte 0xfff5d8d3`850feb8b
                                           VA fff5d8d3850feb8b
PXE at FFFFF6FB7DBEDD88    PPE at FFFFF6FB7DBB1A70    PDE at FFFFF6FB7634E140    PTE at FFFFF6EC69C287F0
contains 0000000000000000
not valid
WARNING: noncanonical VA, accesses will fault !

Code:

Microsoft (R) Windows Debugger Version 6.2.8229.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [D:\Users\Richard\Desktop\MEMORY (15).DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*D:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS Personal
Built by: 7601.17835.amd64fre.win7sp1_gdr.120503-2030
Machine Name:
Kernel base = 0xfffff800`0324c000 PsLoadedModuleList = 0xfffff800`03490670
Debug session time: Wed Jul  4 17:19:25.042 2012 (UTC + 1:00)
System Uptime: 0 days 0:02:25.854
Loading Kernel Symbols
...............................................................
................................................................
.....................................................
Loading User Symbols
PEB is paged out (Peb.Ldr = 00000000`7efdf018).  Type ".hh dbgerr001" for details
Loading unloaded module list
......
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck 3B, {c0000005, fffff800032f0830, fffff8800c5dca10, 0}
Page 3d3b3f not present in the dump file. Type ".hh dbgerr004" for details
Probably caused by : win32k.sys ( win32k!memset+80 )
Followup: MachineOwner
---------
2: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
SYSTEM_SERVICE_EXCEPTION (3b)
An exception happened while executing a system service routine.
Arguments:
Arg1: 00000000c0000005, Exception code that caused the bugcheck
Arg2: fffff800032f0830, Address of the instruction which caused the bugcheck
Arg3: fffff8800c5dca10, Address of the context record for the exception that caused the bugcheck
Arg4: 0000000000000000, zero.
Debugging Details:
------------------

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
FAULTING_IP: 
nt!MiReplenishPageSlist+c0
fffff800`032f0830 f00fba6b1000    lock bts dword ptr [rbx+10h],0
CONTEXT:  fffff8800c5dca10 -- (.cxr 0xfffff8800c5dca10)
rax=04000000002d83fb rbx=bffffa800888bf10 rcx=0000058000000000
rdx=0000000000000050 rsi=0000000000000008 rdi=0000000000000008
rip=fffff800032f0830 rsp=fffff8800c5dd3f0 rbp=fffffa800888d710
 r8=fffff800034fd500  r9=fffffa800cbfa000 r10=fffffa800cbfb358
r11=fffff88003565180 r12=fffff800034fd500 r13=2aaaaaaaaaaaaaab
r14=fdffffffffffffff r15=0000058000000000
iopl=0         nv up ei pl nz na pe nc
cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
nt!MiReplenishPageSlist+0xc0:
fffff800`032f0830 f00fba6b1000    lock bts dword ptr [rbx+10h],0 ds:002b:bffffa80`0888bf20=????????
Resetting default scope
DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT
BUGCHECK_STR:  0x3B
PROCESS_NAME:  iexplore.exe
CURRENT_IRQL:  2
TRAP_FRAME:  fffff8800c5dd8e0 -- (.trap 0xfffff8800c5dd8e0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffff900c23c6000 rbx=0000000000000000 rcx=fffff900c23d4000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffff960000c5090 rsp=fffff8800c5dda78 rbp=fffff80003383a26
 r8=0000000000000020  r9=00000000000002e8 r10=0000000000000034
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
win32k!memset+0x80:
fffff960`000c5090 488911          mov     qword ptr [rcx],rdx ds:fffff900`c23d4000=????????????????
Resetting default scope
LAST_CONTROL_TRANSFER:  from fffff800032eefef to fffff800032f0830
STACK_TEXT:  
fffff880`0c5dd3f0 fffff800`032eefef : fffffa80`0cbfb338 00000000`0000007b fffffa80`08899710 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
fffff880`0c5dd460 fffff800`032db06f : 00000000`00000002 fffff880`00000002 fffff880`02791c00 00000000`00000000 : nt!MiRemoveAnyPage+0x24f
fffff880`0c5dd580 fffff800`032e81ae : 00000000`00000001 fffff900`c23d4000 fffff880`0c5dd8e0 fffff6fc`80611ea0 : nt!MiResolveDemandZeroFault+0x54f
fffff880`0c5dd670 fffff800`032d820b : fffff880`0c5dd4b0 00000000`00000064 00000000`00000064 00000000`00000000 : nt!MiDispatchFault+0x8ce
fffff880`0c5dd780 fffff800`032c92ee : 00000000`00000001 fffff900`c23d4000 00000001`00000000 fffff900`c23c6000 : nt!MmAccessFault+0xe1b
fffff880`0c5dd8e0 fffff960`000c5090 : fffff960`000b3f4e 00000202`0018002b 00000000`00000000 00000000`80000000 : nt!KiPageFault+0x16e
fffff880`0c5dda78 fffff960`000b3f4e : 00000202`0018002b 00000000`00000000 00000000`80000000 fffff880`0c5ddc80 : win32k!memset+0x80
fffff880`0c5dda80 fffff960`000b5458 : fffff800`032ca453 fffff880`0c5dded8 fffff800`03383a26 fffff880`0c5ddc80 : win32k!AllocateObject+0xf2
fffff880`0c5ddac0 fffff960`000076fb : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!SURFMEM::bCreateDIB+0x1f8
fffff880`0c5ddbb0 fffff960`00015c46 : fffff900`c23584e0 fffff900`c0000790 fffff880`0c5dde90 fffff900`c063cbe0 : win32k!psSetupTransparentSrcSurface+0x1a3
fffff880`0c5dde30 fffff960`001926f7 : fffff900`c012b010 00000000`00000000 fffff900`c063cbe0 fffff900`c063cbe0 : win32k!EngAlphaBlend+0x1de
fffff880`0c5de0f0 fffff800`032ca453 : ffffffff`9c010c51 00000000`00000014 00000000`00000000 fffff900`00000618 : win32k!NtGdiAlphaBlend+0x15ff
fffff880`0c5de4f0 00000000`7322059a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`0022c2a8 fffff800`032c2810 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x7322059a
fffff880`0c5de740 fffff900`c084bfe0 : 00000000`000000a0 fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 : nt!KiCallUserMode
fffff880`0c5de748 00000000`000000a0 : fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 : 0xfffff900`c084bfe0
fffff880`0c5de750 fffffa80`0e2b07e0 : fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 : 0xa0
fffff880`0c5de758 fffff960`000c60ce : 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 00460051`00630046 : 0xfffffa80`0e2b07e0
fffff880`0c5de760 003d006c`00720075 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!FreeObject+0x4e
fffff880`0c5de790 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x3d006c`00720075

FOLLOWUP_IP: 
win32k!memset+80
fffff960`000c5090 488911          mov     qword ptr [rcx],rdx
SYMBOL_STACK_INDEX:  6
SYMBOL_NAME:  win32k!memset+80
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: win32k
IMAGE_NAME:  win32k.sys
DEBUG_FLR_IMAGE_TIMESTAMP:  4fb1b20d
STACK_COMMAND:  .cxr 0xfffff8800c5dca10 ; kb
FAILURE_BUCKET_ID:  X64_0x3B_VRF_win32k!memset+80
BUCKET_ID:  X64_0x3B_VRF_win32k!memset+80
Followup: MachineOwner
---------
2: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
SYSTEM_SERVICE_EXCEPTION (3b)
An exception happened while executing a system service routine.
Arguments:
Arg1: 00000000c0000005, Exception code that caused the bugcheck
Arg2: fffff800032f0830, Address of the instruction which caused the bugcheck
Arg3: fffff8800c5dca10, Address of the context record for the exception that caused the bugcheck
Arg4: 0000000000000000, zero.
Debugging Details:
------------------

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
FAULTING_IP: 
nt!MiReplenishPageSlist+c0
fffff800`032f0830 f00fba6b1000    lock bts dword ptr [rbx+10h],0
CONTEXT:  fffff8800c5dca10 -- (.cxr 0xfffff8800c5dca10)
rax=04000000002d83fb rbx=bffffa800888bf10 rcx=0000058000000000
rdx=0000000000000050 rsi=0000000000000008 rdi=0000000000000008
rip=fffff800032f0830 rsp=fffff8800c5dd3f0 rbp=fffffa800888d710
 r8=fffff800034fd500  r9=fffffa800cbfa000 r10=fffffa800cbfb358
r11=fffff88003565180 r12=fffff800034fd500 r13=2aaaaaaaaaaaaaab
r14=fdffffffffffffff r15=0000058000000000
iopl=0         nv up ei pl nz na pe nc
cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
nt!MiReplenishPageSlist+0xc0:
fffff800`032f0830 f00fba6b1000    lock bts dword ptr [rbx+10h],0 ds:002b:bffffa80`0888bf20=????????
Resetting default scope
DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT
BUGCHECK_STR:  0x3B
PROCESS_NAME:  iexplore.exe
CURRENT_IRQL:  2
TRAP_FRAME:  fffff8800c5dd8e0 -- (.trap 0xfffff8800c5dd8e0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffff900c23c6000 rbx=0000000000000000 rcx=fffff900c23d4000
rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
rip=fffff960000c5090 rsp=fffff8800c5dda78 rbp=fffff80003383a26
 r8=0000000000000020  r9=00000000000002e8 r10=0000000000000034
r11=0000000000000000 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
win32k!memset+0x80:
fffff960`000c5090 488911          mov     qword ptr [rcx],rdx ds:fffff900`c23d4000=????????????????
Resetting default scope
LAST_CONTROL_TRANSFER:  from fffff800032eefef to fffff800032f0830
STACK_TEXT:  
fffff880`0c5dd3f0 fffff800`032eefef : fffffa80`0cbfb338 00000000`0000007b fffffa80`08899710 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
fffff880`0c5dd460 fffff800`032db06f : 00000000`00000002 fffff880`00000002 fffff880`02791c00 00000000`00000000 : nt!MiRemoveAnyPage+0x24f
fffff880`0c5dd580 fffff800`032e81ae : 00000000`00000001 fffff900`c23d4000 fffff880`0c5dd8e0 fffff6fc`80611ea0 : nt!MiResolveDemandZeroFault+0x54f
fffff880`0c5dd670 fffff800`032d820b : fffff880`0c5dd4b0 00000000`00000064 00000000`00000064 00000000`00000000 : nt!MiDispatchFault+0x8ce
fffff880`0c5dd780 fffff800`032c92ee : 00000000`00000001 fffff900`c23d4000 00000001`00000000 fffff900`c23c6000 : nt!MmAccessFault+0xe1b
fffff880`0c5dd8e0 fffff960`000c5090 : fffff960`000b3f4e 00000202`0018002b 00000000`00000000 00000000`80000000 : nt!KiPageFault+0x16e
fffff880`0c5dda78 fffff960`000b3f4e : 00000202`0018002b 00000000`00000000 00000000`80000000 fffff880`0c5ddc80 : win32k!memset+0x80
fffff880`0c5dda80 fffff960`000b5458 : fffff800`032ca453 fffff880`0c5dded8 fffff800`03383a26 fffff880`0c5ddc80 : win32k!AllocateObject+0xf2
fffff880`0c5ddac0 fffff960`000076fb : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!SURFMEM::bCreateDIB+0x1f8
fffff880`0c5ddbb0 fffff960`00015c46 : fffff900`c23584e0 fffff900`c0000790 fffff880`0c5dde90 fffff900`c063cbe0 : win32k!psSetupTransparentSrcSurface+0x1a3
fffff880`0c5dde30 fffff960`001926f7 : fffff900`c012b010 00000000`00000000 fffff900`c063cbe0 fffff900`c063cbe0 : win32k!EngAlphaBlend+0x1de
fffff880`0c5de0f0 fffff800`032ca453 : ffffffff`9c010c51 00000000`00000014 00000000`00000000 fffff900`00000618 : win32k!NtGdiAlphaBlend+0x15ff
fffff880`0c5de4f0 00000000`7322059a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`0022c2a8 fffff800`032c2810 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x7322059a
fffff880`0c5de740 fffff900`c084bfe0 : 00000000`000000a0 fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 : nt!KiCallUserMode
fffff880`0c5de748 00000000`000000a0 : fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 : 0xfffff900`c084bfe0
fffff880`0c5de750 fffffa80`0e2b07e0 : fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 : 0xa0
fffff880`0c5de758 fffff960`000c60ce : 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 00460051`00630046 : 0xfffffa80`0e2b07e0
fffff880`0c5de760 003d006c`00720075 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!FreeObject+0x4e
fffff880`0c5de790 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x3d006c`00720075

FOLLOWUP_IP: 
win32k!memset+80
fffff960`000c5090 488911          mov     qword ptr [rcx],rdx
SYMBOL_STACK_INDEX:  6
SYMBOL_NAME:  win32k!memset+80
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: win32k
IMAGE_NAME:  win32k.sys
DEBUG_FLR_IMAGE_TIMESTAMP:  4fb1b20d
STACK_COMMAND:  .cxr 0xfffff8800c5dca10 ; kb
FAILURE_BUCKET_ID:  X64_0x3B_VRF_win32k!memset+80
BUCKET_ID:  X64_0x3B_VRF_win32k!memset+80
Followup: MachineOwner
---------
2: kd> !pte 3d3b3f 
                                           VA 00000000003d3b3f
PXE at FFFFF6FB7DBED000    PPE at FFFFF6FB7DA00000    PDE at FFFFF6FB40000008    PTE at FFFFF68000001E98
contains 00700002E6D5B867  contains 02F00002E3B5F867  contains 01800002E5869867  contains CDE00003BAA94025
pfn 2e6d5b    ---DA--UWEV  pfn 2e3b5f    ---DA--UWEV  pfn 2e5869    ---DA--UWEV  pfn 3baa94    ----A--UR-V
2: kd> !cmkd.stack
Call Stack : 28 frames
## Stack-Pointer    Return-Address   Call-Site       
00 fffff8800c5dc148 fffff800032ca769 nt!KeBugCheckEx+0 
01 fffff8800c5dc150 fffff800032ca0bc nt!KiBugCheckDispatch+69 
02 fffff8800c5dc290 fffff800032f5e2d nt!KiSystemServiceHandler+7c 
03 fffff8800c5dc2d0 fffff800032f4c05 nt!RtlpExecuteHandlerForException+d 
04 fffff8800c5dc300 fffff80003305b81 nt!RtlDispatchException+415 
05 fffff8800c5dc9e0 fffff800032ca842 nt!KiDispatchException+135 
06 fffff8800c5dd080 fffff800032c914a nt!KiExceptionDispatch+c2 
07 fffff8800c5dd260 fffff800032f0830 nt!KiGeneralProtectionFault+10a 
08 fffff8800c5dd3f0 fffff800032eefef nt!MiReplenishPageSlist+c0 (perf)
09 fffff8800c5dd460 fffff800032db06f nt!MiRemoveAnyPage+24f 
0a fffff8800c5dd580 fffff800032e81ae nt!MiResolveDemandZeroFault+54f 
0b fffff8800c5dd670 fffff800032d820b nt!MiDispatchFault+8ce 
0c fffff8800c5dd780 fffff800032c92ee nt!MmAccessFault+e1b 
0d fffff8800c5dd8e0 fffff960000c5090 nt!KiPageFault+16e 
0e fffff8800c5dda78 fffff960000b3f4e win32k!memset+80 
0f fffff8800c5dda80 fffff960000b5458 win32k!AllocateObject+f2 
10 fffff8800c5ddac0 fffff960000076fb win32k!SURFMEM::bCreateDIB+1f8 
11 fffff8800c5ddbb0 fffff96000015c46 win32k!psSetupTransparentSrcSurface+1a3 
12 fffff8800c5dde30 fffff960001926f7 win32k!EngAlphaBlend+1de 
13 fffff8800c5de0f0 fffff800032ca453 win32k!NtGdiAlphaBlend+15ff 
14 fffff8800c5de4f0 000000007322059a nt!KiSystemServiceCopyEnd+13 
2: kd> !vtop 3baa94 00000000003d3b3f
Amd64VtoP: Virt 00000000`003d3b3f, pagedir 3baa94
Amd64VtoP: PML4E 3baa94
Amd64VtoP: PML4E read error 0x80004002
Virtual address 3d3b3f translation fails, error 0x80004002.
2: kd> !thread
THREAD fffffa80101ed6d0  Cid 1d0c.1d10  Teb: 000000007efdb000 Win32Thread: fffff900c2d1cc20 RUNNING on processor 2
Not impersonating
DeviceMap                 fffff8a002fdeae0
Owning Process            fffffa800fdffb30       Image:         iexplore.exe
Attached Process          N/A            Image:         N/A
Wait Start TickCount      9349           Ticks: 0
Context Switch Count      2899           IdealProcessor: 1                 LargeStack
UserTime                  00:00:00.124
KernelTime                00:00:00.218
Win32 Start Address 0x0000000000ba28a0
Stack Init fffff8800c5de6f0 Current fffff8800c5dde80
Base fffff8800c5df000 Limit fffff8800c5d6000 Call fffff8800c5de740
Priority 11 BasePriority 8 UnusualBoost 0 ForegroundBoost 2 IoPriority 2 PagePriority 5
Child-SP          RetAddr           : Args to Child                                                           : Call Site
fffff880`0c5dc148 fffff800`032ca769 : 00000000`0000003b 00000000`c0000005 fffff800`032f0830 fffff880`0c5dca10 : nt!KeBugCheckEx
fffff880`0c5dc150 fffff800`032ca0bc : fffff880`0c5dd1b8 fffff880`0c5dca10 00000000`00000000 fffff960`002a2490 : nt!KiBugCheckDispatch+0x69
fffff880`0c5dc290 fffff800`032f5e2d : fffff960`002f0ba0 fffff960`002c01a4 fffff960`00000000 fffff880`0c5dd1b8 : nt!KiSystemServiceHandler+0x7c
fffff880`0c5dc2d0 fffff800`032f4c05 : fffff800`03412638 fffff880`0c5dc348 fffff880`0c5dd1b8 fffff800`0324c000 : nt!RtlpExecuteHandlerForException+0xd
fffff880`0c5dc300 fffff800`03305b81 : fffff880`0c5dd1b8 fffff880`0c5dca10 fffff880`00000000 00000000`00000008 : nt!RtlDispatchException+0x415
fffff880`0c5dc9e0 fffff800`032ca842 : fffff880`0c5dd1b8 bffffa80`0888bf10 fffff880`0c5dd260 00000000`00000008 : nt!KiDispatchException+0x135
fffff880`0c5dd080 fffff800`032c914a : 00000000`00000001 fffff960`00749f85 00000000`00000001 fffff880`0c5dd480 : nt!KiExceptionDispatch+0xc2
fffff880`0c5dd260 fffff800`032f0830 : 00000000`00000001 00000000`0000001a fffff880`03565180 00000000`00000001 : nt!KiGeneralProtectionFault+0x10a (TrapFrame @ fffff880`0c5dd260)
fffff880`0c5dd3f0 fffff800`032eefef : fffffa80`0cbfb338 00000000`0000007b fffffa80`08899710 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
fffff880`0c5dd460 fffff800`032db06f : 00000000`00000002 fffff880`00000002 fffff880`02791c00 00000000`00000000 : nt!MiRemoveAnyPage+0x24f
fffff880`0c5dd580 fffff800`032e81ae : 00000000`00000001 fffff900`c23d4000 fffff880`0c5dd8e0 fffff6fc`80611ea0 : nt!MiResolveDemandZeroFault+0x54f
fffff880`0c5dd670 fffff800`032d820b : fffff880`0c5dd4b0 00000000`00000064 00000000`00000064 00000000`00000000 : nt!MiDispatchFault+0x8ce
fffff880`0c5dd780 fffff800`032c92ee : 00000000`00000001 fffff900`c23d4000 00000001`00000000 fffff900`c23c6000 : nt!MmAccessFault+0xe1b
fffff880`0c5dd8e0 fffff960`000c5090 : fffff960`000b3f4e 00000202`0018002b 00000000`00000000 00000000`80000000 : nt!KiPageFault+0x16e (TrapFrame @ fffff880`0c5dd8e0)
fffff880`0c5dda78 fffff960`000b3f4e : 00000202`0018002b 00000000`00000000 00000000`80000000 fffff880`0c5ddc80 : win32k!memset+0x80
fffff880`0c5dda80 fffff960`000b5458 : fffff800`032ca453 fffff880`0c5dded8 fffff800`03383a26 fffff880`0c5ddc80 : win32k!AllocateObject+0xf2
fffff880`0c5ddac0 fffff960`000076fb : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!SURFMEM::bCreateDIB+0x1f8
fffff880`0c5ddbb0 fffff960`00015c46 : fffff900`c23584e0 fffff900`c0000790 fffff880`0c5dde90 fffff900`c063cbe0 : win32k!psSetupTransparentSrcSurface+0x1a3
fffff880`0c5dde30 fffff960`001926f7 : fffff900`c012b010 00000000`00000000 fffff900`c063cbe0 fffff900`c063cbe0 : win32k!EngAlphaBlend+0x1de
fffff880`0c5de0f0 fffff800`032ca453 : ffffffff`9c010c51 00000000`00000014 00000000`00000000 fffff900`00000618 : win32k!NtGdiAlphaBlend+0x15ff
fffff880`0c5de4f0 00000000`7322059a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`0c5de560)
00000000`0022c2a8 fffff800`032c2810 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x7322059a
fffff880`0c5de740 fffff900`c084bfe0 : 00000000`000000a0 fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 : nt!KiCallUserMode
fffff880`0c5de748 00000000`000000a0 : fffffa80`0e2b07e0 fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 : 0xfffff900`c084bfe0
fffff880`0c5de750 fffffa80`0e2b07e0 : fffff960`000c60ce 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 : 0xa0
fffff880`0c5de758 fffff960`000c60ce : 00000000`00000000 fffff880`0c5dec70 00430030`003d0064 00460051`00630046 : 0xfffffa80`0e2b07e0
fffff880`0c5de760 003d006c`00720075 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32k!FreeObject+0x4e
fffff880`0c5de790 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x3d006c`00720075
2: kd> dt nt!_MMPFN fffff800`032f0830
   +0x000 u1               : <unnamed-tag>
   +0x008 u2               : <unnamed-tag>
   +0x010 PteAddress       : 0xfff5d8d3`850feb8b _MMPTE
   +0x010 VolatilePteAddress : 0xfff5d8d3`850feb8b Void
   +0x010 Lock             : 0n-2062554229
   +0x010 PteLong          : 0xfff5d8d3`850feb8b
   +0x018 u3               : <unnamed-tag>
   +0x01c UsedPageTableEntries : 0x20
   +0x01e VaType           : 0x1 ''
   +0x01f ViewCount        : 0xf ''
   +0x020 OriginalPte      : _MMPTE
   +0x020 AweReferenceCount : 0n-170333820
   +0x028 u4               : <unnamed-tag>
2: kd> !pte 0xfff5d8d3`850feb8b
                                           VA fff5d8d3850feb8b
PXE at FFFFF6FB7DBEDD88    PPE at FFFFF6FB7DBB1A70    PDE at FFFFF6FB7634E140    PTE at FFFFF6EC69C287F0
contains 0000000000000000
not valid
WARNING: noncanonical VA, accesses will fault !

Relevant parts:

Code:

4: kd> !pte 0xfff5d8d3`850feb8b
                                           VA [COLOR=#ff0000][B]fff5d8d3850feb8b[/B][/COLOR]
PXE at FFFFF6FB7DBEDD88    PPE at FFFFF6FB7DBB1A70    PDE at FFFFF6FB7634E140    PTE at FFFFF6EC69C287F0
contains 0000000000000000
not valid
WARNING: noncanonical VA, accesses will fault !

Code:

2: kd> !pte 0xfff5d8d3`850feb8b
                                           VA [COLOR=#ff0000][B]fff5d8d3850feb8b[/B][/COLOR]
PXE at FFFFF6FB7DBEDD88    PPE at FFFFF6FB7DBB1A70    PDE at FFFFF6FB7634E140    PTE at FFFFF6EC69C287F0
contains 0000000000000000
not valid
WARNING: noncanonical VA, accesses will fault !

Any idea why the Virtual Address in these two completely different dumps is exactly the same? Any idea what it means? Is it normal? I assume it is just some system thing which will of course be the same in each dump, or something mundane and useless like that.

Thanks a lot.

Vir Gnarus · Jul 9, 2012

I personally can't tell you what's going on there, but I can tell you that you're most likely deducing things backwards. As in you go for PTE first, which will then provide you the pfn. View the Windbg manual for !pfn for an example of this.

Also, why are you checking page info for the instruction that caused the fault? What you're checking isn't the bad memory that's addressed, but rather the memory that stores the instruction in the nt module that performed memory operation (the instruction being at nt!MiReplenishPageSlist+c0). To simplify, you're looking at the memory storing the nt module code, not the bad memory referenced. If you want the latter, you'll have to look at the code itself, which is referencing [rbx+10]. Look at rbx register, add 10, and you got your bad memory reference (the r command shows the memory referenced for convenience). Evidently, from the looks of it, we're dealing with a missing bit (the 'b' as the highest byte gives it away). Try correcting it (by replacing "b" with "f" in the address) and work on verifying the memory address from there.

Btw, what I do find curious is that both crashdumps appear to have faulted by addressing the same exact memory, which makes me personally think we're dealing with software or driver that's causing it. Here's where I found the two:

Code:

[I]First dump:[/I]

STACK_TEXT:
fffff880`0c344dd8 fffff800`03324d88 : 00000000`0000001e ffffffff`c0000005 fffff800`032ff830 00000000`00000000 : nt!KeBugCheckEx
 fffff880`0c344de0 fffff800`032d9842 : fffff880`0c3455b8 [COLOR=#ff0000]bffffa80`0888bf10[/COLOR] fffff880`0c345660 00000000`00000006 : nt! ?? ::FNODOBFM::`string'+0x48d3d
 fffff880`0c345480 fffff800`032d814a : fffffa80`0fe7bbb0 fffff8a0`006a6650 fffffa80`0d9b1430 fffffa80`0deb4180 : nt!KiExceptionDispatch+0xc2
 fffff880`0c345660 fffff800`032ff830 : ffffffff`ffffffff fffffa80`0f2dfb50 00000000`18030000 fffff800`03302106 : nt!KiGeneralProtectionFault+0x10a
 fffff880`0c3457f0 fffff800`032fdfef : fffffa80`0cbfb338 00000000`0000007b fffffa80`0889c710 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
 fffff880`0c345860 fffff800`032e7614 : 00000000`00000000 00000000`00000002 00000000`00000000 ffffffff`ffffffff : nt!MiRemoveAnyPage+0x24f
 fffff880`0c345980 fffff800`032d82ee : 00000000`00000001 00000000`18037000 00000000`16822101 00000000`000000b0 : nt!MmAccessFault+0x1224
 fffff880`0c345ae0 00000000`6ec06af7 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x16e
 00000000`1f05d000 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x6ec06af7

[I]Second dump:

[/I] CONTEXT:  
fffff8800c5dca10 -- (.cxr 0xfffff8800c5dca10) 
rax=04000000002d83fb rbx=[COLOR=#ff0000]bffffa800888bf10[/COLOR] rcx=0000058000000000 
rdx=0000000000000050 rsi=0000000000000008 rdi=0000000000000008 
rip=fffff800032f0830 rsp=fffff8800c5dd3f0 rbp=fffffa800888d710  
r8=fffff800034fd500  r9=fffffa800cbfa000 r10=fffffa800cbfb358 
r11=fffff88003565180 r12=fffff800034fd500 r13=2aaaaaaaaaaaaaab 
r14=fdffffffffffffff r15=0000058000000000 
iopl=0         nv up ei pl nz na pe nc 
cs=0010  ss=0018  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202 
nt!MiReplenishPageSlist+0xc0: fffff800`032f0830 f00fba6b1000    lock bts dword ptr [[COLOR=#ff0000]rbx+10h[/COLOR]],0 ds:002b:[COLOR=#ff0000]bffffa80`0888bf20[/COLOR]=????????

As an extra clue, both crashdumps report iexplore.exe as the running process (and therefore the faulting thread associated with it) at the time.

niemiro · Jul 14, 2012

Thanks a lot for your help here. I did wonder whether I was doing something like that, and to be honest, I didn't know 100% what I was doing.

It is great that I am able to learn from you in this way.

Do you think that you would be able to get anything out of these dumps if I uploaded the kernel memory dump? Do you have any other ideas on how to track this problem down? Verifier hasn't turned up anything useful, nor have any of the hundreds of BSODs been more informative than any of the others.

I have reduced the number of active, 3rd party drivers from 60 to 27, and can't push that much further, although I will try to.

Thanks a lot for your help.

niemiro · Jul 14, 2012

BTW, would I get any better results out of a live kernel debug, bearing in mind that I don't actually have quite the right gear at the moment.

The target (Windows 7 x64) has an internal 1394a port, which could be connected to a host (Vista x86) 1394a.

Unfortunately, I don't have a suitable firewire cable at the moment (although I could easily purchase one), and the host has a bad graphics card, but if I am lucky, it might just stay up long enough to get some useful data. lol.

niemiro · Jul 14, 2012

Have gone really drastic, and have cut it down to 17, and updated two others. I really won't be able to cut out many more. Will see how it goes.

niemiro · Jul 14, 2012

Another two gone. Down to 15... - exactly 25% of normal. EDIT: Now 14.

We shall have to wait and see now.

EDIT: Just so you can see the difference:

Initially:

Read More:

Now:

Read More:

niemiro · Jul 14, 2012

And it crashed again, even after taking all of those drivers out. Unless anyone has any other ideas, I might have to remove my discrete graphics card, and temporarily move to onboard video. I wouldn't put it past ATI drivers to cause this.

niemiro · Jul 14, 2012

Aha!

Is this conclusive proof? I only just learned about this today.

My graphics card driver is on the latest version, and is a driver only install (no CCC).

Code:

dps KiPreBugcheckStackSaveArea KiPreBugcheckStackSaveArea+3000

...

fffff800`034be208  fffff800`0347a260 nt!MiSpecialPool
fffff800`034be210  fffff800`03384df3 nt!RtlWalkFrameChain+0x63
fffff800`034be218  00000000`00000000
fffff800`034be220  fffff800`0375bdee nt!VfDeadlockDeleteMemoryRange+0x2e
fffff800`034be228  00000000`00000020
fffff800`034be230  00000000`00000000
fffff800`034be238  00000000`0000000d
fffff800`034be240  fffff800`03385c7b nt!RtlCaptureStackBackTrace+0x4b
fffff800`034be248  00000000`00000003
fffff800`034be250  fffffa80`0d2ea298
fffff800`034be258  00000000`00000000
fffff800`034be260  fffff800`00000004
fffff800`034be268  00000000`c0000000
fffff800`034be270  fffff980`1f5a2ff0
fffff800`034be278  fffff980`16bd0fe0
fffff800`034be280  00000000`00000000
fffff800`034be288  fffff980`16bd0fe0
fffff800`034be290  fffff980`105caec0
fffff800`034be298  fffff880`0a271540
fffff800`034be2a0  fffff800`03761870 nt!VfRemLockDeleteMemoryRange+0x30
fffff800`034be2a8  00000000`00000020
fffff800`034be2b0  fffff880`0a272000
fffff800`034be2b8  00000000`000000c4
fffff800`034be2c0  00000000`00000000
fffff800`034be2c8  ffffc49f`fc1c0e38
fffff800`034be2d0  fffff800`033644a2 nt!MmQuerySpecialPoolBlockType+0x42
fffff800`034be2d8  00000000`c0000000
fffff800`034be2e0  fffff980`1f5a2ff0
fffff800`034be2e8  fffff980`16bd0fe0
fffff800`034be2f0  00000000`00000000
fffff800`034be2f8  00000000`00000000
fffff800`034be300  fffff800`033f593b nt!ExDeferredFreePool+0xf33
fffff800`034be308  fffff800`0324c000 nt!KiSelectNextThread <PERF> (nt+0x0)
fffff800`034be310  00000000`5958504b
fffff800`034be318  00000000`00002b14
fffff800`034be320  fffffa80`0d2ea118
fffff800`034be328  00000000`00000010
fffff800`034be330  00000000`00000002
fffff800`034be338  00000000`00000008
fffff800`034be340  fffff800`0375321e nt!ExFreePoolSanityChecks+0x4e
fffff800`034be348  00000000`c0000001
fffff800`034be350  00000000`00000000
fffff800`034be358  fffff8a0`07abc770
fffff800`034be360  00000000`00000000
fffff800`034be368  00000000`00000000
fffff800`034be370  00000000`00000000
fffff800`034be378  fffff980`16bd0fe0
fffff800`034be380  fffff800`0376b8ef nt!VerifierExFreePoolWithTag+0x2f
fffff800`034be388  fffff980`16bd0fe0
fffff800`034be390  fffff980`0151c540
fffff800`034be398  fffff980`105caec0
fffff800`034be3a0  fffff880`0a271540
fffff800`034be3a8  00000000`00000000
[COLOR=#ff0000][B]fffff800`034be3b0  fffff880`03209d9e*** ERROR: Module load completed but symbols could not be loaded for atikmpag.sys
 atikmpag+0x9d9e[/B][/COLOR]
fffff800`034be3b8  00000000`00000000
fffff800`034be3c0  fffff800`0375bdee nt!VfDeadlockDeleteMemoryRange+0x2e
fffff800`034be3c8  00000000`00000040
fffff800`034be3d0  00000000`00000000
fffff800`034be3d8  00000000`00000001
fffff800`034be3e0  fffff8a0`038b2910
fffff800`034be3e8  fffff980`1f5a2ff0
fffff800`034be3f0  fffffa80`0d2ea198
fffff800`034be3f8  00000000`00000000
fffff800`034be400  00000000`00000002
fffff800`034be408  fffff8a0`038b2910
fffff800`034be410  00000000`00000000
fffff800`034be418  fffff8a0`07a78570
fffff800`034be420  00000000`00000004
fffff800`034be428  fffff8a0`07a78560
fffff800`034be430  00000000`00000003
fffff800`034be438  00000000`00000040
fffff800`034be440  fffff800`03761870 nt!VfRemLockDeleteMemoryRange+0x30
fffff800`034be448  00000000`00000040
fffff800`034be450  fffff880`0a272000
fffff800`034be458  00000000`00000002
fffff800`034be460  00000000`00000001
fffff800`034be468  00000000`00000040
fffff800`034be470  fffff800`03762945 nt!VfFreePoolNotification+0x55
fffff800`034be478  fffff8a0`07a78560
fffff800`034be480  00000000`00000040
fffff800`034be488  00000000`00000003
fffff800`034be490  00000000`00000010
fffff800`034be498  fffff8a0`07a78560
fffff800`034be4a0  fffff800`033f6fbd nt!ExFreePoolWithTag+0x22d
fffff800`034be4a8  fffff8a0`038b2910
fffff800`034be4b0  00000000`00000000
fffff800`034be4b8  00000000`00000000
fffff800`034be4c0  00000000`00000001
fffff800`034be4c8  fffff880`0a2715b0
fffff800`034be4d0  fffff880`048ce0d1 dxgkrnl!DXGADAPTER::AcquireDdiSync+0xc9
fffff800`034be4d8  fffff8a0`038b2910
fffff800`034be4e0  00000000`00000000
fffff800`034be4e8  fffff8a0`07abc770
fffff800`034be4f0  00000000`00000000
fffff800`034be4f8  00000000`00000000
fffff800`034be500  fffff880`0a2715b0
fffff800`034be508  fffff980`105caec0
fffff800`034be510  fffff880`04902e79 dxgkrnl!DXGADAPTER::DdiCloseAllocation+0x4d
fffff800`034be518  fffff8a0`038b2900
fffff800`034be520  fffffa80`0ed8e000
fffff800`034be528  00000000`00000000
fffff800`034be530  fffff980`16b6c700
fffff800`034be538  00000000`00000000
fffff800`034be540  00000000`00000000
fffff800`034be548  fffff8a0`079a6000
fffff800`034be550  fffff880`04904d4e dxgkrnl!DXGDEVICE::DestroyAllocations+0x2ca
fffff800`034be558  00000000`00000000
fffff800`034be560  00000000`00000000
fffff800`034be568  00000000`00000001
fffff800`034be570  00000000`00000174
fffff800`034be578  00000000`00000001
fffff800`034be580  fffff8a0`038b2910
fffff800`034be588  00000000`00000001
fffff800`034be590  fffff800`033f5c57 nt!ExDeferredFreePool+0x1283
fffff800`034be598  fffff8a0`07c94790
fffff800`034be5a0  fffffa80`00000000
fffff800`034be5a8  00000000`00000000
fffff800`034be5b0  fffff800`032d42c2 nt!ExReleaseResourceAndLeaveCriticalRegion+0x12

...

usasma · Jul 14, 2012

Probably not conclusive - but it seems to definitely place the blame on something video.

I always suspect the software that switches between the 2 graphics cards (VIRTU on some systems) as it seems to have problems staying stable.

niemiro · Jul 14, 2012

usasma said:
Probably not conclusive - but it seems to definitely place the blame on something video.

I always suspect the software that switches between the 2 graphics cards (VIRTU on some systems) as it seems to have problems staying stable.

Thanks a lot, John. That is a good point about Virtu (which mine is). Although I don't actively use it, I will just make sure that it is properly disabled.

Thanks again.

usasma · Jul 14, 2012

We've seen quite a few issues with VIRTU over the last month or two - but there haven't been any significant one's recently. This tells me that the most recent VIRTU update is probably the most stable.

Gotta wonder about conflicts between the graphics cards if both are active. Can you disable one of them and test just the other?

Remember the big hullaballoo about HP loading both AMD and Intel processor drivers on all their systems? It worked fine for them for years - until Microsoft made an update that assumed (rightly so) that there'd only be one processor driver active. And that caused HP's around the world to BSOD! I wonder about the same issues with any set of drivers being loaded that do the same thing (such as the Intel and the ATI video drivers).

Patrick · Jul 15, 2012

:wave:Just to post and give you some hope, hopefully not false!

I was plagued with 116 BSOD's for months, and that's what actually got me into BSOD analysis, so people didn't have to go through what I did and get all the headaches that come with it. I replaced every component imaginable by RMA'ing and did several clean Windows installs, and was still BSOD'ing. I was going to lose my mind, it was draining me and putting so much unnecessary stress on me. satrow recommended that I rollback from CCC 12.3 / whatever version I was trying, to a version I remember working, which was 12.1 for my 5850s. Well, I said rather than rolling back, I'll just go ahead and do a clean Windows install with it, and install 12.1 straight away. This way if I get issues, I have no idea what it could be. Sure enough.... BSOD free for months now.

I had never had an issue with AMD drivers before. I was one of those guys that figured if you were having an issue, it was PEBKAC and you went wrong somewhere... but it turns out it's true in one way or another that their drivers can sometimes be a bit shady. I guess it all depends on whether or not AMD tested your card generation on the latest drivers. I figured they kind of stopped caring about the 5xxx series, that's why I was having issues when going higher in driver version.

Who knows

niemiro · Jul 15, 2012

Thanks a lot for your encouragement, E-Peen :)

I might actually roll back to that particular version, and see if it is stable for me too, if nothing else helps.

However, the plot thickens considerable. Read on!

I just got another BSOD, and checked the StackSaveArea again:

Code:

fffff800`034cd208  fffff800`03488260 nt!MiSpecialPool
fffff800`034cd210  fffff800`03392df3 nt!RtlWalkFrameChain+0x63
fffff800`034cd218  00000000`00000000
fffff800`034cd220  fffff800`03769dee nt!VfDeadlockDeleteMemoryRange+0x2e
fffff800`034cd228  00000000`00000330
fffff800`034cd230  00000000`00000000
fffff800`034cd238  00000000`0000000d
fffff800`034cd240  fffff800`03393c7b nt!RtlCaptureStackBackTrace+0x4b
fffff800`034cd248  00000000`00000003
fffff800`034cd250  fffffa80`0ce4e098
fffff800`034cd258  00000000`00000000
fffff800`034cd260  00000000`00000001
fffff800`034cd268  00000000`00000190
fffff800`034cd270  fffffa80`0e225000
fffff800`034cd278  fffff980`218dacd0
fffff800`034cd280  fffffa80`0e237480
fffff800`034cd288  fffff980`218dacd0
fffff800`034cd290  fffff980`218dacd0
fffff800`034cd298  00000000`00000192
fffff800`034cd2a0  fffff800`0376f870 nt!VfRemLockDeleteMemoryRange+0x30
fffff800`034cd2a8  00000000`00000330
fffff800`034cd2b0  fffff800`0376ca01 nt!VfDeadlockAcquireResource+0x81
fffff800`034cd2b8  00000000`00000000
fffff800`034cd2c0  fffff880`04c4a39f*** ERROR: Module load completed but symbols could not be loaded for atikmdag.sys
 atikmdag+0x3a39f
fffff800`034cd2c8  ffffc405`7a382f10
fffff800`034cd2d0  fffff800`033724a2 nt!MmQuerySpecialPoolBlockType+0x42
fffff800`034cd2d8  00000000`00000190
fffff800`034cd2e0  fffffa80`0e225000
fffff800`034cd2e8  fffff980`218dacd0
fffff800`034cd2f0  fffffa80`0e237480
fffff800`034cd2f8  00000000`00000000
fffff800`034cd300  fffff800`0340393b nt!ExDeferredFreePool+0xf33
fffff800`034cd308  fffff800`0325a000 nt!KiSelectNextThread <PERF> (nt+0x0)
fffff800`034cd310  00000000`6264444e
fffff800`034cd318  ffffffff`ffeb5590
fffff800`034cd320  fffffa80`0eb65bb8
fffff800`034cd328  00000000`00000000
fffff800`034cd330  fffffa80`0e9bb010
fffff800`034cd338  00000000`00000000
fffff800`034cd340  fffff800`0376121e nt!ExFreePoolSanityChecks+0x4e
fffff800`034cd348  00000000`0e19d400
fffff800`034cd350  00000000`00000000
fffff800`034cd358  fffffa80`0e9bb001
fffff800`034cd360  00000000`00000000
fffff800`034cd368  00000000`00000000
fffff800`034cd370  00000000`00000000
fffff800`034cd378  fffff980`218dacd0
fffff800`034cd380  fffff800`037798ef nt!VerifierExFreePoolWithTag+0x2f
fffff800`034cd388  00000000`00000000
fffff800`034cd390  fffff980`03afef80
fffff800`034cd398  fffff980`218dacd0
fffff800`034cd3a0  00000000`00000192
fffff800`034cd3a8  fffff980`1ea08e60
fffff800`034cd3b0  fffff800`0377992b nt!VerifierExFreePool+0x1b
fffff800`034cd3b8  fffff980`1ea08e60
fffff800`034cd3c0  fffffa80`0e225000
fffff800`034cd3c8  00000000`00000000
fffff800`034cd3d0  fffffa80`0e237480
fffff800`034cd3d8  fffff980`03afaf60
fffff800`034cd3e0  fffff880`018ab8a7 ndis!NdisMFreeNetBufferSGList+0xa7
fffff800`034cd3e8  fffff980`1ea08e60
fffff800`034cd3f0  00000000`00000192
fffff800`034cd3f8  fffffa80`0d6b0bf8
fffff800`034cd400  fffff880`1026f569*** ERROR: Module load completed but symbols could not be loaded for e1c62x64.sys
 e1c62x64+0x2c569
fffff800`034cd408  fffffa80`0e225000
fffff800`034cd410  fffff980`03ae2e20
fffff800`034cd418  00000000`00000000
fffff800`034cd420  fffff880`1026cc91 e1c62x64+0x29c91
fffff800`034cd428  00000000`00000000
fffff800`034cd430  fffffa80`0d6b0bf8
fffff800`034cd438  00000000`00000000
fffff800`034cd440  fffff880`1026f663 e1c62x64+0x2c663
fffff800`034cd448  00000000`00000000
fffff800`034cd450  fffffa80`0e225040
fffff800`034cd458  00000000`00000000
fffff800`034cd460  fffffa80`0e225000
fffff800`034cd468  fffffa80`0e228180
fffff800`034cd470  00000000`00000000
fffff800`034cd478  00000100`00000001
fffff800`034cd480  fffffa80`0e225000
fffff800`034cd488  00000000`00000000
fffff800`034cd490  00000000`00000000
fffff800`034cd498  fffff800`00ba2ea0
fffff800`034cd4a0  fffff980`03a68f00
fffff800`034cd4a8  00000000`00000000
fffff800`034cd4b0  00000000`00000000
fffff800`034cd4b8  fffff980`03a68ff0
fffff800`034cd4c0  fffff880`10260e57 e1c62x64+0x1de57
fffff800`034cd4c8  fffff980`03a04ff0
fffff800`034cd4d0  fffff980`00000000
fffff800`034cd4d8  fffffa80`00000000
fffff800`034cd4e0  00000000`00000000
fffff800`034cd4e8  fffffa80`00000000
fffff800`034cd4f0  fffffa80`00000000
fffff800`034cd4f8  fffffa80`0ea94b88
fffff800`034cd500  fffffa80`0fc9ab50
fffff800`034cd508  fffff800`00ba2e10
fffff800`034cd510  fffff880`04cf5425 atikmdag+0xe5425
fffff800`034cd518  fffffa80`0ea94000
fffff800`034cd520  fffffa80`0ea94b88
fffff800`034cd528  00000000`00000000
fffff800`034cd530  fffffa80`0fc9ab50
fffff800`034cd538  00000000`00000000
fffff800`034cd540  fffffa80`0eaa4390
fffff800`034cd548  00000000`00000000
fffff800`034cd550  00000000`00000000
fffff800`034cd558  fffff800`00ba2db0
fffff800`034cd560  00000000`00000004
fffff800`034cd568  ffff0080`1092fcfa
fffff800`034cd570  00000000`00000000
fffff800`034cd578  00000000`00000000
fffff800`034cd580  00000000`00000000
fffff800`034cd588  fffff980`03a68ff0
fffff800`034cd590  00000000`00000000
fffff800`034cd598  ffff0001`00000000
fffff800`034cd5a0  fffffa80`0e225000
fffff800`034cd5a8  00000000`00000000
fffff800`034cd5b0  fffff880`1025f35c e1c62x64+0x1c35c

So...both my networking drivers, and my graphics card drivers (curiously, I re-ran an old dump, and the stack was corrupt, but a repeating string of "Local Area Connection")

Anyway, I actually suspected my networking driver a long time ago. If I was doing something on the internet (refreshing a page, opening IE, etc., it was WAY more likely to crash. The rest could be accounted for by background networking activity)

So, I removed my networking drivers, and ran without internet. BSODs easily reduced 10 fold, but still continued, ever so occasionally (I am only talking about the 0x1E and 0x3B stack trash here, the rest were easy, and were solved months ago). But it still continued. I have tried loads of different versions, and found the version I am currently using (not the most recent), reduces, but does not stop, the BSODs. It is the best so far.

Now, as mentioned above, I also suspect the graphics card driver. Now this dump blames them both (ish - I agree that one could easily call the other, and only one actually caused the crash).

Any more thoughts?

Richard

Patrick · Jul 15, 2012

If you haven't tried already, I would recommend checking DevMan and making sure there are no power saving features enabled for the network adapter. If there are, that may be a contributing issue. It would explain the network drivers for showing up as culprits, and it may be chain reacting and somehow setting off the AMD drivers.

Vir Gnarus · Jul 16, 2012

That's a very interesting find, neimiro, I just hope that you aren't going off on a rabbit trail with it. It does appear relevant, though. I'll have to look into it.

niemiro · Jul 16, 2012

Vir Gnarus said:
That's a very interesting find, neimiro, I just hope that you aren't going off on a rabbit trail with it. It does appear relevant, though. I'll have to look into it.

Yeah, it looks to be quite useful, if only I knew how to wield it properly! I found it in Windows Internals 5th Edition p1146-1147 (Stack Trash). I don't really mind if it turns into a wild goose chase. It is all a great learning experience and I don't mind making mistakes and learning from them, until you and everyone else gets fed up with pushing me back onto the right track! Just let me know and I will seal my lips :)

niemiro · Jul 16, 2012

Oh, and it doesn't work in minidumps

EDIT: I just found an old dump (not mine) which actually blamed it. Curious.

FAILURE_BUCKET_ID: X64_0x1E_c0000005_nt!KiPreBugcheckStackSaveArea+3f5e

I can guess a few reasons, but can't test any as I don't even have a minidump

niemiro · Aug 13, 2012

Well...the BSODs continue. However, they are being pretty infrequent at the moment (sometimes I go a month without a BSOD, other times I get >25 in a day - most fail to leave even a minidump, so you don't know about them, but I do!)

Anyway, I just got another one today. Still MiReplenishPageSlist (IIRC), still an Access Violation, still same memory address, still same IRQL, similar call-stack, but it seems much less corrupt than the previous dumps.

One thing I do notice is that the call stack between the minidump and full memory dump is completely different. I am uploading the memory dump now, but it will take a while. Minidump attached. Output from full memory dump pasted below:

Code:

Microsoft (R) Windows Debugger Version 6.2.8229.0 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Windows\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*D:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS Personal
Built by: 7601.17835.amd64fre.win7sp1_gdr.120503-2030
Machine Name:
Kernel base = 0xfffff800`03258000 PsLoadedModuleList = 0xfffff800`0349c670
Debug session time: Mon Aug 13 14:52:17.569 2012 (UTC + 1:00)
System Uptime: 0 days 0:24:58.381
Loading Kernel Symbols
...............................................................
................................................................
...........................
Loading User Symbols
PEB is paged out (Peb.Ldr = 000007ff`fffdf018).  Type ".hh dbgerr001" for details
Loading unloaded module list
......
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck D1, {fffff88001749c08, 2, 8, fffff88001749c08}
Probably caused by : Ntfs.sys ( Ntfs! ?? ::NNGAKEGL::`string'+2bf0 )
Followup: MachineOwner
---------
6: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: fffff88001749c08, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000008, value 0 = read operation, 1 = write operation
Arg4: fffff88001749c08, address which referenced memory
Debugging Details:
------------------

READ_ADDRESS:  fffff88001749c08 
CURRENT_IRQL:  2
FAULTING_IP: 
Ntfs! ?? ::NNGAKEGL::`string'+2bf0
fffff880`01749c08 55              push    rbp
DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT
BUGCHECK_STR:  0xD1
PROCESS_NAME:  explorer.exe
TRAP_FRAME:  fffff8800a773fa0 -- (.trap 0xfffff8800a773fa0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=04000000002c37fb rbx=0000000000000000 rcx=0000058000000000
rdx=0000000000005114 rsi=0000000000000000 rdi=0000000000000000
rip=fffff800032fc830 rsp=fffff8800a774130 rbp=fffffa80084a9710
 r8=fffff80003509500  r9=0000000000000000 r10=fffffa800c8ea758
r11=fffff880034fb180 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
nt!MiReplenishPageSlist+0xc0:
fffff800`032fc830 f00fba6b1000    lock bts dword ptr [rbx+10h],0 ds:00000000`00000010=????????
Resetting default scope
EXCEPTION_RECORD:  fffff8800a773ef8 -- (.exr 0xfffff8800a773ef8)
ExceptionAddress: fffff800032fc830 (nt!MiReplenishPageSlist+0x00000000000000c0)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 0000000000000000
   Parameter[1]: ffffffffffffffff
Attempt to read from address ffffffffffffffff
LAST_CONTROL_TRANSFER:  from fffff800032d6769 to fffff800032d71c0
FAILED_INSTRUCTION_ADDRESS: 
Ntfs! ?? ::NNGAKEGL::`string'+2bf0
fffff880`01749c08 55              push    rbp
STACK_TEXT:  
fffff880`0a772cb8 fffff800`032d6769 : 00000000`0000000a fffff880`01749c08 00000000`00000002 00000000`00000008 : nt!KeBugCheckEx
fffff880`0a772cc0 fffff800`032d53e0 : fffff8a0`2202fc01 fffff880`0a7739a0 fffff980`4a683940 fffff880`016aa840 : nt!KiBugCheckDispatch+0x69
fffff880`0a772e00 fffff880`01749c08 : fffff800`033023ac 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiPageFault+0x260
fffff880`0a772f98 fffff800`033023ac : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : Ntfs! ?? ::NNGAKEGL::`string'+0x2bf0
fffff880`0a772fa0 fffff800`03301e2d : fffff880`016aa834 fffff880`0a774630 00000000`00000000 fffff880`0165a000 : nt!_C_specific_handler+0x8c
fffff880`0a773010 fffff800`03300c05 : fffff880`016aa834 fffff880`0a773088 fffff880`0a773ef8 fffff880`0165a000 : nt!RtlpExecuteHandlerForException+0xd
fffff880`0a773040 fffff800`03311b81 : fffff880`0a773ef8 fffff880`0a773750 fffff880`00000000 00000000`0000000a : nt!RtlDispatchException+0x415
fffff880`0a773720 fffff800`032d6842 : fffff880`0a773ef8 bffffa80`084a7f10 fffff880`0a773fa0 00000000`00000006 : nt!KiDispatchException+0x135
fffff880`0a773dc0 fffff800`032d514a : 00000000`0000b74f fffff800`032e7bdc fffff780`c0000000 00000000`00000000 : nt!KiExceptionDispatch+0xc2
fffff880`0a773fa0 fffff800`032fc830 : f8a025ae`66300400 f8a025ae`66380400 f8a025ae`66400400 f8a025ae`66480400 : nt!KiGeneralProtectionFault+0x10a
fffff880`0a774130 fffff800`032fafef : fffffa80`0c8ea738 00000000`0000007b fffffa80`084a1f10 00000000`0000007b : nt!MiReplenishPageSlist+0xc0
fffff880`0a7741a0 fffff800`03302eab : 00000000`00000000 ffffffff`00000000 ffffffff`ffffffff 00000000`000000f6 : nt!MiRemoveAnyPage+0x24f
fffff880`0a7742c0 fffff800`032ff76e : fffff980`486be000 00000000`0a03e040 fffff880`00000000 00000000`00001000 : nt!MmCopyToCachedPage+0xa0b
fffff880`0a7744b0 fffff800`032ffd24 : fffffa80`107214f0 00000000`0a03e040 fffff880`0a7745f0 fffff880`00000000 : nt!CcMapAndCopyInToCache+0x20e
fffff880`0a7745a0 fffff880`016f6c18 : 00000000`0a9c0000 fffffa80`106e6400 fffff880`0a774690 fffffa80`00010000 : nt!CcCopyWrite+0x194
fffff880`0a774630 fffff880`01412102 : fffffa80`106e6410 fffff880`014154f2 00000000`00010000 00000000`00010001 : Ntfs!NtfsCopyWriteA+0x208
fffff880`0a774830 fffff880`014158ba : fffff880`0a774900 fffffa80`0d9f8698 00000000`0a030000 00000000`00010000 : fltmgr!FltpPerformFastIoCall+0xf2
fffff880`0a774890 fffff880`0143383e : 00000000`00010000 00000000`00000000 fffffa80`106e6410 fffff880`0a774a00 : fltmgr!FltpPassThroughFastIo+0xda
fffff880`0a7748d0 fffff800`035e56de : fffffa80`106e6484 fffffa80`00000002 fffffa80`0d2958f0 fffffa80`106e6484 : fltmgr!FltpFastIoWrite+0x1ce
fffff880`0a774970 fffff800`032d6453 : 00000000`00008001 00000000`0000078c 00000000`00000000 00000000`11eeced8 : nt!NtWriteFile+0x5ad
fffff880`0a774a70 00000000`7723139a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
00000000`0f97cc28 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x7723139a

STACK_COMMAND:  kb
FOLLOWUP_IP: 
Ntfs! ?? ::NNGAKEGL::`string'+2bf0
fffff880`01749c08 55              push    rbp
SYMBOL_STACK_INDEX:  3
SYMBOL_NAME:  Ntfs! ?? ::NNGAKEGL::`string'+2bf0
FOLLOWUP_NAME:  MachineOwner
MODULE_NAME: Ntfs
IMAGE_NAME:  Ntfs.sys
DEBUG_FLR_IMAGE_TIMESTAMP:  4d79997b
FAILURE_BUCKET_ID:  X64_0xD1_VRF_CODE_AV_BAD_IP_Ntfs!_??_::NNGAKEGL::_string_+2bf0
BUCKET_ID:  X64_0xD1_VRF_CODE_AV_BAD_IP_Ntfs!_??_::NNGAKEGL::_string_+2bf0
Followup: MachineOwner
---------


6: kd> u fffff88001749c08
Ntfs! ?? ::NNGAKEGL::`string'+0x2bf0:
fffff880`01749c08 55              push    rbp
fffff880`01749c09 4883ec30        sub     rsp,30h
fffff880`01749c0d 488bea          mov     rbp,rdx
fffff880`01749c10 488b01          mov     rax,qword ptr [rcx]
fffff880`01749c13 8b08            mov     ecx,dword ptr [rax]
fffff880`01749c15 ff150d5ff5ff    call    qword ptr [Ntfs!_imp_FsRtlIsNtstatusExpected (fffff880`0169fb28)]
fffff880`01749c1b 33c9            xor     ecx,ecx
fffff880`01749c1d 84c0            test    al,al

Also, this is the only dump ever to have combined the NtfsCopyWriteA BSOD with the 0x1E BSOD. It is a very curious crash, this particular one, IMO.

Please help :)

niemiro · Aug 13, 2012

Full memory dump now available here, if you think it will help: http://www.mediafire.com/?f976mvu7f0aulyv

Thank you :)

jcgriff2 · Aug 13, 2012

Do you have an SSD or HDD??

ntfs.sys named as probable cause.

https://www.sysnative.com/drivers/driver.php?id=Ntfs.sys

Also noted in the dump was a 0xc0000005 exception = memory access violation.

ntfs is very unusual to see as p/c on a 0xd1 bugcheck; 0xc5 exception not uncommon at all.

Have you tested your HDD (or updated firmware on SSD), whichever is applicable?

Or tried another HDD?

[SOLVED] A whole range of BSoDs

Senior Administrator, Windows Update Expert

BSOD Kernel Dump Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Retired Admin

Senior Administrator, Windows Update Expert

Retired Admin

Sysnative Staff

Senior Administrator, Windows Update Expert

Sysnative Staff

BSOD Kernel Dump Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Senior Administrator, Windows Update Expert

Attachments

Senior Administrator, Windows Update Expert

Co-Founder / AdminBSOD Instructor/ExpertMicrosoft MVP (Ret.)

Co-Founder / Admin
BSOD Instructor/Expert
Microsoft MVP (Ret.)