Windows Fails or BSODs When Updating, SFC Fails (OS: Windows 10.0.18362 Build 18362)

Sketch · Jul 29, 2023

Hello, I have tried a number of times over the years to get my windows update to work without success. Unfortunately I no longer remember when it stopped working, so I can't be sure if it was shortly after some system change or another. I have tried many different things, and the last time I attempted this a number of months ago I wrote down a bunch of them, so in no particular order:
- consolidating c drive (now has tons of space, I used to partition the c drive to try and keep a small portion for just windows stuff)
- running SFC (fails at 64%, have tried fixing it separately but no dice)
- running SFCFix.exe
- DISM in online mode
- DISM off of an installation iso
- running chkdsk
- in place install w/updates from installation media (both usb an iso)
- in place install w/o updates from installation media
- running a registry checker
- manually checking registry values related to c drive paths (though I'm only lightly familiar with windows registry values so I might have missed something wrong there)
- running a repair tool that came with the registry checker
- running the windows update troubleshooter
- checking for any obviously bad drivers, and checking for updates on intel graphics, nvidia, monitors, ssds/hdds

I even tried addressing some of the issues noted by SFC in the CBS log such as security overlaps, but to no avail.

I'm hoping someone here with more in-depth knowledge can help me fix this. I don't like the thought that I'm probably behind on security updates, and I'm missing some nice features like DX12 support, some programs don't support Windows 10 before a certain version, etc.

I don't recall what error windows update would spit out when it would fail before getting to the install phase, but I do know that when it BSODs during the install phase it reports DRIVER_PNP_WATCHDOG as the stop code. If I get it to error again before the install phase I'll post the code. I'm attaching the CBS.zip, as well as output from dism/sfc, SFCfix, and ComponentScanner.

Maxstar · Aug 6, 2023

Hi and welcome to Sysnative,

Can you please provide the following files if they exist.
- %SystemRoot%\MEMORY.DMP
- %SystemRoot%\Minidump\*.dmp
- %SystemDrive%\$Windows.~BT\Sources
- %SystemRoot%\INF\setupapi.dev.log

ZIP these files and upload the file to www.wetransfer.com and post the link in your next reply.

Sketch · Aug 6, 2023

Of course, here they are. I will note the minidumps are from April, I don't believe I tried updating at that time, but I have included them just in case. There were also several setupapi.dev files, so I've included them, with the unmarked one being from 8/5/2023. The files are zipped here: MEMORY.zip

x BlueRobot · Aug 7, 2023

Unfortunately, the MEMORY.DMP has been corrupted so I can't do anything with it unfortunately. The crashes from April were a Stop 0x101 and Stop 0x50, both don't particularly tell us anything useful but let's eliminate the possibly of there being any hardware failures particularly your RAM.

Please download and run MemTest86 for at least 8 passes, this will run two complete runs of 4 passes.

Test RAM with PassMark MemTest86 (version 7.4 was used)

Sketch said:
- checking for any obviously bad drivers,

How did you check? Using Driver Verifier?

Sketch · Aug 7, 2023

I legitimately do not remember, it's been a while now. I grabbed that list from when I was talking to my friend about all the stuff I had tried. I think I might have just manually walked through device manager and tried to get any info on the various drivers listed there, looking for updated versions, if not finding them running the option to check for new drivers in windows. I'll run driver verifier to double check that as well as memtest when I get a chance today.

Sketch · Aug 7, 2023

Oh well, I decided to go ahead and get driver verifier running before I got back into work, and it got my computer in a BSOD loop.
Stop Code: DRIVER_VERIFIER_IOMANAGER_VIOLATION
What failed: Wdf01000.sys

Maxstar · Aug 8, 2023

Hi,

Please provide the latest dump files.
- %SystemRoot%\MEMORY.DMP
- %SystemRoot%\Minidump\*.dmp

Sketch · Aug 8, 2023

I'm waiting for the memory test program to finish at the moment, but I'll upload them as soon as it's done.

Sketch · Aug 8, 2023

Okay so first the mem test and then the dumps:

1 ) I ran the first run of 4 passes and that finished with no errors. I'll run it a proper set tomorrow as that's all I have time to do at the moment (my PC is in my room so I don't want to run it all night and keep myself awake), but I wanted to give it a go tonight and see how long it would take.

2 ) I was not able to find a MEMORY.dmp file in %SystemRoot%. I even tried searching with "all subfolders" enabled just to make sure. Are there actions I could have performed that would have caused it to be cleared? I *do* have a minidump from the time of the BSOD however. That's attached to this post.

x BlueRobot · Aug 8, 2023

Sketch said:
Are there actions I could have performed that would have caused it to be cleared?

It's probably because of your dump file settings, the default used to produce a single MEMORY.DMP which was overwritten with each crash and then keep the last 5 minidumps. They seem to have changed it now so the default is to only produce minidumps. Let's double check with this reg query:

Code:

reg query "HKLM\SYSTEM\CurrentControlSet\Control\CrashControl"

It looks like your vmulti.sys driver is causing problems here. The Sysnative DRT describes it as a virtual joystick driver? Any ideas on what it could be related to?

Rich (BB code):

7: kd> !irp ffffbd8798944d80
Irp is active with 5 stacks 4 is current (= 0xffffbd8798944f28)
 No Mdl: No System Buffer: Thread ffffbd878cd28080:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [N/A(0), N/A(0)]
            0  0 00000000 00000000 00000000-00000000    

            Args: 00000000 00000000 00000000 00000000
 [N/A(0), N/A(0)]
            0 10 00000000 00000000 00000000-00000000    

            Args: 00000000 00000000 00000000 00000000
 [IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
            0 e0 ffffbd878f7d9b30 00000000 fffff80042170530-ffffbd8798944f28 Success Error Cancel 
           \Driver\vmulti    nt!IovpInternalCompletionTrap
            Args: 00000001 00000000 00000000 00000000
>[IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
            0  0 ffffbd8792fcd060 00000000 00000000-00000000    
           \Driver\mshidkmdf
            Args: 00000001 00000000 00000000 00000000
 [IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
            0  0 ffffbd8792fcd060 00000000 00000000-00000000    
           \Driver\mshidkmdf
            Args: 00000001 00000000 00000000 00000000

In case you're wondering why the Wdf01000.sys driver was blamed originally, it was because the vmulti.sys driver is written with the WDF (Windows Driver Framework) and that driver is responsible for handling most of the framework operations for the driver. It's why we now recommend that DV be enabled on certain Microsoft drivers.

Rich (BB code):

7: kd> ln fffff80042c1aac0

 [minkernel\wdf\framework\shared\core\fxdevice.cpp @ 1368] (fffff800`42c1aac0)   Wdf01000!FxDevice::DispatchWithLock   |  (fffff800`42c1adc0)   Wdf01000!FxSystemWorkItem::_WorkItemThunk
Exact matches:
    Wdf01000!FxDevice::DispatchWithLock (struct _DEVICE_OBJECT *, struct _IRP *)

Rich (BB code):

7: kd> knL
 # Child-SP          RetAddr               Call Site
00 ffff8087`857a0dc8 fffff800`421706b3     nt!KeBugCheckEx
01 ffff8087`857a0dd0 fffff800`42177917     nt!VerifierBugCheckIfAppropriate+0xdf
02 ffff8087`857a0e10 fffff800`41b2a44f     nt!ViErrorFinishReport+0x117
03 ffff8087`857a0e70 fffff800`421775bd     nt!ViErrorReport1+0x63
04 ffff8087`857a0f10 fffff800`42182178     nt!VfErrorReport1+0x9
05 ffff8087`857a0f40 fffff800`421773ec     nt!VfPnpVerifyIrpStackUpward+0x158
06 ffff8087`857a0fa0 fffff800`4216ff53     nt!VfMajorVerifyIrpStackUpward+0x74
07 ffff8087`857a0ff0 fffff800`421658f6     nt!IovpCompleteRequest2+0xe3
08 ffff8087`857a1060 fffff800`41849079     nt!IovpLocalCompletionRoutine+0x96
09 ffff8087`857a10c0 fffff800`42165315     nt!IopfCompleteRequest+0x119
0a ffff8087`857a11d0 fffff800`419fc515     nt!IovCompleteRequest+0x1e1
0b ffff8087`857a12c0 fffff800`42b80195     nt!IofCompleteRequest+0x1b35e5
0c ffff8087`857a12f0 fffff800`42169450     VerifierExt!IofCompleteRequest_wrapper+0x145
0d ffff8087`857a1340 fffff803`fc3b12e8     nt!VerifierIofCompleteRequest+0x10 << The driver completes the I/O when it shouldn't have and is caught by Driver Verifier
0e ffff8087`857a1370 ffffbd87`98944ee0     vmulti+0x12e8 << Crash here!
0f ffff8087`857a1378 00000000`00000288     0xffffbd87`98944ee0
10 ffff8087`857a1380 ffffbd87`98944ee0     0x288
11 ffff8087`857a1388 ffff867c`2a800000     0xffffbd87`98944ee0
12 ffff8087`857a1390 ffffbd87`976d4d00     0xffff867c`2a800000
13 ffff8087`857a1398 fffff800`42c1acad     0xffffbd87`976d4d00
14 (Inline Function) --------`--------     Wdf01000!PreprocessIrp+0x2d
15 (Inline Function) --------`--------     Wdf01000!DispatchWorker+0x178
16 (Inline Function) --------`--------     Wdf01000!FxDevice::Dispatch+0x196
17 ffff8087`857a13a0 fffff800`4197fd8a     Wdf01000!FxDevice::DispatchWithLock+0x1ed << I/O request received here and passed down to vmulti.sys
18 ffff8087`857a1400 fffff800`421650a9     nt!IopfCallDriver+0x56
19 ffff8087`857a1440 fffff800`42172ac8     nt!IovCallDriver+0x275
1a ffff8087`857a1480 fffff803`fc3c61d7     nt!VerifierIofCallDriver+0x18
1b ffff8087`857a14b0 fffff803`fc3d3939     mshidkmdf!HidKmdfPnp+0x97
1c ffff8087`857a14e0 fffff803`fc3fbaf1     HIDCLASS!HidpCallDriver+0xb9
1d ffff8087`857a1550 fffff803`fc3fa2a1     HIDCLASS!HidpFdoPnp+0x141
1e ffff8087`857a15c0 fffff803`fc3d25e5     HIDCLASS!HidpIrpMajorPnp+0x71
1f ffff8087`857a1600 fffff800`4197fd8a     HIDCLASS!HidpMajorHandler+0x1b5
20 ffff8087`857a1690 fffff800`421650a9     nt!IopfCallDriver+0x56
21 ffff8087`857a16d0 fffff800`419f5887     nt!IovCallDriver+0x275
22 ffff8087`857a1710 fffff800`41e3da64     nt!IofCallDriver+0x1c20c7
23 ffff8087`857a1750 fffff800`41f0b1da     nt!IopSynchronousCall+0xf8
24 ffff8087`857a17d0 fffff800`41f0b0bc     nt!PnpIrpQueryID+0x56
25 ffff8087`857a1860 fffff800`41efc0a9     nt!PnpQueryID+0x34
26 ffff8087`857a18c0 fffff800`41ef8699     nt!PipProcessStartPhase3+0x165
27 ffff8087`857a19a0 fffff800`41f83768     nt!PipProcessDevNodeTree+0x375
28 ffff8087`857a1a60 fffff800`4195d0fe     nt!PiProcessStartSystemDevices+0x60
29 ffff8087`857a1ab0 fffff800`418beee5     nt!PnpDeviceActionWorker+0x45e
2a ffff8087`857a1b70 fffff800`4192be95     nt!ExpWorkerThread+0x105
2b ffff8087`857a1c10 fffff800`419c930a     nt!PspSystemThreadStartup+0x55
2c ffff8087`857a1c60 00000000`00000000     nt!KiStartSystemThread+0x2a

Sketch · Aug 8, 2023

Here's the output from the reg query:

Code:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl
    AutoReboot    REG_DWORD    0x1
    CrashDumpEnabled    REG_DWORD    0x7
    DumpFile    REG_EXPAND_SZ    %SystemRoot%\MEMORY.DMP
    LogEvent    REG_DWORD    0x1
    MinidumpDir    REG_EXPAND_SZ    %SystemRoot%\Minidump
    MinidumpsCount    REG_DWORD    0x5
    Overwrite    REG_DWORD    0x1
    DumpFilters    REG_MULTI_SZ    dumpfve.sys

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl\FullLiveKernelReports
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl\LiveKernelReports
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl\StorageTelemetry

CrashDumpEnabled is 7, which as far as I can tell seems to be a flag for "AutomaticMemoryDump": Automatic Memory Dump - Windows drivers
Digging a little deeper and the only lead I had was that it interacts with the paging file size. Years ago I had windows on a fairly small partition so I thought I might have manually override the paging file size, but nope, it's automatic (and ~20GB at that). Maybe there's something in there you can see that I'm missing.

As to vmulti.sys, I'm not 100% sure. I looked around and found an old github for it, it looks like it handles several kinds of devices "Virtual Multiple HID Driver (multitouch, mouse, digitizer, keyboard, joystick)" : GitHub - djpnewton/vmulti: Virtual Multiple HID Driver (multitouch, mouse, digitizer, keyboard, joystick)
I'm trying to think back about what new hardware I've plugged in over the years from when Windows Update started giving problems, and rule out things that were plugged in after. Possible culprits: An old Wacom Intuos tablet and pen, a wireless mouse, and a logitech USB controller (I think model F310?). That said if it's unrelated to the Windows Update stuff and is new, then that would include several more mice over the years, a Corsair keyboard, and a Huion Kamvas Pro 16 tablet /pen.

What do you mean by "enable DV on certain windows drivers"?

x BlueRobot · Aug 8, 2023

Sketch said:
Maybe there's something in there you can see that I'm missing.

How much free disk space do you have on your Windows drive? From what I remember, there has to be a minimum % free for it to write a MEMORY.DMP. You probably have more than enough space though.

Sketch said:
What do you mean by "enable DV on certain windows drivers"?

Driver Verifier; most people recommend that you don't enable it on Windows drivers and we used to do too, however, I came across an OSR article which recommended that you enable it on: ndis.sys, Wdf01000.sys, fltmgr.sys and storport.sys. Since some drivers can seemingly "pass" Driver Verifier due to most of their functionality being handled by a Windows driver.

Analyst’s Perspective: My Driver Passes Driver Verifier! (Or Does It…)

Let's just disable that vmulti.sys driver for now using FRST:

FRST Fix
NOTICE: This script was written specifically for this user, for use on that particular machine. Running this on another machine may cause damage to your operating system
1. Please download Farbar Recovery Scan Tool and save it to your Desktop.
Note: You need to run the 64-bit Version so please ensure you download that one.
2. Download the attached fixlist.txt and save it to the Desktop.
Note. It's important that both files, FRST64 and fixlist.txt are in the same location or the fix will not work (in this case...the desktop).
3. Run FRST64 by Right-Clicking on the file and choosing Run as administrator.
4. Press the Fix button just once and wait. If for some reason the tool needs a restart, please make sure you let the system restart normally. After that let the tool complete its run.
5. When finished FRST64 will generate a log on the Desktop (Fixlog.txt). Please post the contents of it in your reply.

Sketch · Aug 8, 2023

Oh woah, this thing had a ton of space last I checked. I have 24GB of RAM so if it's dumping all of it then yeah, it wouldn't fit. I'm busy with work at the moment, but I'll try cleaning this up and then messing with the drivers later tonight.

Sketch · Aug 8, 2023

Okay, went and found a game steam had installed to C: instead of my usual drive, so that freed up another 120GB. I also ran both iterations of mem test this time around and still no errors. I just ran FRST64 with both it and the fix list on the desktop as admin and so far all the devices I've tested seem to still work fine. I've attached the fix log below.

Sketch · Aug 8, 2023

I also ran driver verifier for a bit just now. Was able to boot into windows no issues other than everything running slow (presumably because the verifier is running?). Anyway I'll attach the /query output in case it's relevant at all.

x BlueRobot · Aug 9, 2023

That all looks good, I would disable Driver Verifier for now, it probably won't find anything. I haven't looked at your CBS logs yet so I'll let @Maxstar decide on the next steps.

Maxstar · Aug 9, 2023

Thanks @x BlueRobot for helping with the BSOD issue...

@Sketch I would suggest to attempt to update again, if it fails attach a new copy of the CBS logs and or the *.dmp files when another BSOD appears.

Sketch · Aug 9, 2023

Alright, I'll do that today when I get a moment.

x BlueRobot · Aug 9, 2023

Okay, please let us know how it goes, hopefully you won't have any issues.

Sketch · Aug 9, 2023

Alright it just finished rolling back, I was doing some cleaning while letting it update and missed the specific error, but checked in time to see the "undoing changes" screen. Looking at windows related errors in the event viewer brings up a few things though:

Code:

    Date       Time            Source                 Event ID Category
1) 8/9/2023 2:28:03 PM    DistributedCOM               10005   None
2) 8/9/2023 2:27:00 PM    WindowsUpdateClient          20      Windows Update Agent
3) 8/9/2023 2:25:44 PM    volmgr                       46      None
4) 8/9/2023 1:57:07 PM    Perflib                      1020    None
5) 8/9/2023 1:54:19 PM    VSS                          8193    None
6) 8/9/2023 1:54:08 PM    CAPI2                        513     None
7) 8/9/2023 1:54:07 PM    CAPI2                        513     None
8) 8/9/2023 1:50:41 PM    Windows Remote Management    10142   None

The messages when clicking on them are:

Code:

1) DCOM got error "15612" attempting to start the service GamingServices with arguments "Unavailable" in order to run the server: {3E8C9ABE-9226-4609-BF5B-60288A391DEE}

2) Installation Failure: Windows failed to install the following update with error 0xC1900101: Feature update to Windows 10, version 22H2.

3) Crash dump initialization failed!

4) The required buffer size is greater than the buffer size passed to the Collect function of the "C:\Windows\System32\perfts.dll" Extensible Counter DLL for the "LSM" service. The given buffer size was 19176 and the required size was 39112.

5) Volume Shadow Copy Service error: Unexpected error calling routine QueryFullProcessImageNameW.  hr = 0x80070006, The handle is invalid.
. 

Operation:
   Executing Asynchronous Operation

Context:
   Current State: DoSnapshotSet

6) Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.

Details:
AddCoreCsiFiles : RtlConvertNtFilePathToWin32Path() failed.

System Error:
0xC0000039 (unresolvable).

7) Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.

Details:
AddCoreCsiFiles : RtlConvertNtFilePathToWin32Path() failed.

System Error:
0xC0000039 (unresolvable).

8) The WinRM service cannot migrate the listener with Address * and Transport HTTP. A listener that has the same Address and Transport configuration already exists.

These are all roughly from around the time I restarted the computer to finish the update installation.
I'm guessing #3 is why I can't find a minidump or memoery.dmp from today, the 9th. There is one of each from when I ran the driver verifier last night though, which is weird because I don't remember it crashing after disabling that one driver. And to clarify, I did disable DriverVerifier after my post last night.

Anyway I'll attach the minidump/memory.dmp from last night in case it's of use. It's available here: DriverVerifier_8-8-8pm.zip

I'm also attaching the latest CBS.log to this post.

Sorry if this is a bit of an info dump. I want to try and grab as much as info as I can for y'all since I think we're operating in different time zones.

Windows Fails or BSODs When Updating, SFC Fails (OS: Windows 10.0.18362 Build 18362)

Contributor

Attachments

Moderator, Windows Update Expert, Security Analyst

Contributor

Administrator

Contributor

Contributor

Moderator, Windows Update Expert, Security Analyst

Contributor

Contributor

Attachments

Administrator

Contributor

Administrator

Attachments

Contributor

Contributor

Attachments

Contributor

Attachments

Administrator

Moderator, Windows Update Expert, Security Analyst

Contributor

Administrator

Contributor

Attachments