- May 7, 2013
- 10,400
Rich (BB code):
DRIVER_VERIFIER_IOMANAGER_VIOLATION (c9)
The IO manager has caught a misbehaving driver.
Arguments:
Arg1: 000000000000022e, The caller has completed a successful IRP_MJ_PNP instead of passing it down.
Arg2: fffff80042c1aac0, The address in the driver's code where the error was detected.
Arg3: ffffbd8798944d80, IRP address.
Arg4: 0000000000000000This bugcheck was a good example of why you should enable Driver Verifier on a select number of Microsoft drivers. The driver in question was actually written using WDF framework so many of it's calls will be through this framework driver. The second parameter perfectly demonstrates this fact. Notice how the problematic function is for WDF?
Rich (BB code):
7: kd> ln fffff80042c1aac0
Browse module
Set bu breakpoint
[minkernel\wdf\framework\shared\core\fxdevice.cpp @ 1368] (fffff800`42c1aac0) Wdf01000!FxDevice::DispatchWithLock | (fffff800`42c1adc0) Wdf01000!FxSystemWorkItem::_WorkItemThunk
Exact matches:
Wdf01000!FxDevice::DispatchWithLock (struct _DEVICE_OBJECT *, struct _IRP *)If you look at what the function does, then we will see that it will complete the IRP:
Rich (BB code):
NTSTATUS
FxDevice::DispatchWithLock(
__in MdDeviceObject DeviceObject,
__in MdIrp Irp
)
{
NTSTATUS status;
FxIrp irp(Irp);
switch (_RequiresRemLock(irp.GetMajorFunction(),
irp.GetMinorFunction())) {
case FxDeviceRemLockRequired:
status = Mx::MxAcquireRemoveLock(
&_GetFxWdmExtension(DeviceObject)->IoRemoveLock,
Irp
);
if (!NT_SUCCESS(status)) {
irp.SetStatus(status);
irp.CompleteRequest(IO_NO_INCREMENT);
return status;
}
break;
case FxDeviceRemLockOptIn:
status = _AcquireOptinRemoveLock(
DeviceObject,
Irp
);
if (!NT_SUCCESS(status)) {
irp.SetStatus(status);
irp.CompleteRequest(IO_NO_INCREMENT);
return status;
}
break;
case FxDeviceRemLockTestValid:
//
// Try to Acquire and Release the RemLock. If acquiring the lock
// fails then it is not safe to process the IRP and the IRP should
// be completed immediately.
//
status = Mx::MxAcquireRemoveLock(
&_GetFxWdmExtension(DeviceObject)->IoRemoveLock,
Irp
);
if (!NT_SUCCESS(status)) {
irp.SetStatus(status);
irp.CompleteRequest(IO_NO_INCREMENT);
return status;
}
Mx::MxReleaseRemoveLock(
&_GetFxWdmExtension(DeviceObject)->IoRemoveLock,
Irp
);
break;
}
return Dispatch(DeviceObject, Irp);
}The IRP itself reveals the type of PnP request were handling at the time of the crash, along with the problematic driver. We can see that our driver completes the request and then passes it down to the next driver. All PnP IRPs must be completed by the bus driver associated to the device node, it is not permissible for any other driver to complete this kind of request.
Rich (BB code):
7: kd> !irp ffffbd8798944d80
Irp is active with 5 stacks 4 is current (= 0xffffbd8798944f28)
No Mdl: No System Buffer: Thread ffffbd878cd28080: Irp stack trace.
cmd flg cl Device File Completion-Context
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 10 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
0 e0 ffffbd878f7d9b30 00000000 fffff80042170530-ffffbd8798944f28 Success Error Cancel
\Driver\vmulti nt!IovpInternalCompletionTrap << Driver Verifier function for verifying a request hasn't been prematurely completed
Args: 00000001 00000000 00000000 00000000
>[IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
0 0 ffffbd8792fcd060 00000000 00000000-00000000
\Driver\mshidkmdf
Args: 00000001 00000000 00000000 00000000
[IRP_MJ_PNP(1b), IRP_MN_QUERY_ID(13)]
0 0 ffffbd8792fcd060 00000000 00000000-00000000
\Driver\mshidkmdf
Args: 00000001 00000000 00000000 00000000
Irp Extension present at 0xffffbd8798944fb8:The call stack further confirms our findings from earlier:
Rich (BB code):
7: kd> knL
# Child-SP RetAddr Call Site
00 ffff8087`857a0dc8 fffff800`421706b3 nt!KeBugCheckEx
01 ffff8087`857a0dd0 fffff800`42177917 nt!VerifierBugCheckIfAppropriate+0xdf
02 ffff8087`857a0e10 fffff800`41b2a44f nt!ViErrorFinishReport+0x117
03 ffff8087`857a0e70 fffff800`421775bd nt!ViErrorReport1+0x63
04 ffff8087`857a0f10 fffff800`42182178 nt!VfErrorReport1+0x9
05 ffff8087`857a0f40 fffff800`421773ec nt!VfPnpVerifyIrpStackUpward+0x158
06 ffff8087`857a0fa0 fffff800`4216ff53 nt!VfMajorVerifyIrpStackUpward+0x74
07 ffff8087`857a0ff0 fffff800`421658f6 nt!IovpCompleteRequest2+0xe3
08 ffff8087`857a1060 fffff800`41849079 nt!IovpLocalCompletionRoutine+0x96
09 ffff8087`857a10c0 fffff800`42165315 nt!IopfCompleteRequest+0x119
0a ffff8087`857a11d0 fffff800`419fc515 nt!IovCompleteRequest+0x1e1
0b ffff8087`857a12c0 fffff800`42b80195 nt!IofCompleteRequest+0x1b35e5
0c ffff8087`857a12f0 fffff800`42169450 VerifierExt!IofCompleteRequest_wrapper+0x145
0d ffff8087`857a1340 fffff803`fc3b12e8 nt!VerifierIofCompleteRequest+0x10
0e ffff8087`857a1370 ffffbd87`98944ee0 vmulti+0x12e8
0f ffff8087`857a1378 00000000`00000288 0xffffbd87`98944ee0
10 ffff8087`857a1380 ffffbd87`98944ee0 0x288
11 ffff8087`857a1388 ffff867c`2a800000 0xffffbd87`98944ee0
12 ffff8087`857a1390 ffffbd87`976d4d00 0xffff867c`2a800000
13 ffff8087`857a1398 fffff800`42c1acad 0xffffbd87`976d4d00
14 (Inline Function) --------`-------- Wdf01000!PreprocessIrp+0x2d
15 (Inline Function) --------`-------- Wdf01000!DispatchWorker+0x178
16 (Inline Function) --------`-------- Wdf01000!FxDevice::Dispatch+0x196
17 ffff8087`857a13a0 fffff800`4197fd8a Wdf01000!FxDevice::DispatchWithLock+0x1ed << Crash here!
18 ffff8087`857a1400 fffff800`421650a9 nt!IopfCallDriver+0x56
19 ffff8087`857a1440 fffff800`42172ac8 nt!IovCallDriver+0x275
1a ffff8087`857a1480 fffff803`fc3c61d7 nt!VerifierIofCallDriver+0x18
1b ffff8087`857a14b0 fffff803`fc3d3939 mshidkmdf!HidKmdfPnp+0x97
1c ffff8087`857a14e0 fffff803`fc3fbaf1 HIDCLASS!HidpCallDriver+0xb9
1d ffff8087`857a1550 fffff803`fc3fa2a1 HIDCLASS!HidpFdoPnp+0x141
1e ffff8087`857a15c0 fffff803`fc3d25e5 HIDCLASS!HidpIrpMajorPnp+0x71
1f ffff8087`857a1600 fffff800`4197fd8a HIDCLASS!HidpMajorHandler+0x1b5
20 ffff8087`857a1690 fffff800`421650a9 nt!IopfCallDriver+0x56
21 ffff8087`857a16d0 fffff800`419f5887 nt!IovCallDriver+0x275
22 ffff8087`857a1710 fffff800`41e3da64 nt!IofCallDriver+0x1c20c7
23 ffff8087`857a1750 fffff800`41f0b1da nt!IopSynchronousCall+0xf8
24 ffff8087`857a17d0 fffff800`41f0b0bc nt!PnpIrpQueryID+0x56
25 ffff8087`857a1860 fffff800`41efc0a9 nt!PnpQueryID+0x34
26 ffff8087`857a18c0 fffff800`41ef8699 nt!PipProcessStartPhase3+0x165
27 ffff8087`857a19a0 fffff800`41f83768 nt!PipProcessDevNodeTree+0x375
28 ffff8087`857a1a60 fffff800`4195d0fe nt!PiProcessStartSystemDevices+0x60
29 ffff8087`857a1ab0 fffff800`418beee5 nt!PnpDeviceActionWorker+0x45e
2a ffff8087`857a1b70 fffff800`4192be95 nt!ExpWorkerThread+0x105
2b ffff8087`857a1c10 fffff800`419c930a nt!PspSystemThreadStartup+0x55
2c ffff8087`857a1c60 00000000`00000000 nt!KiStartSystemThread+0x2aAdditionally, to add to our evidence of vmulti.sys being the suspected driver, if you check the device object being passed to the last IoCallDriver function before the crash, then you will notice that it belongs to vmulti.sys:
Rich (BB code):
7: kd> !devobj ffffbd87`8f7d9b30
fffff80041c47a38: Unable to get value of ObpRootDirectoryObject
Device object (ffffbd878f7d9b30) is for:
InfoMask field not found for _OBJECT_HEADER at ffffbd878f7d9b00
\Driver\vmulti DriverObject ffffbd878f7addd0
Current Irp 00000000 RefCount 0 Type 00000004 Flags 00000000
SecurityDescriptor ffffa886b0e28be0 DevExt ffffbd87976d4ff0 DevObjExt ffffbd878f7d9ca8
ExtensionFlags (0xf0000000) DOE_RAW_FDO, DOE_BOTTOM_OF_FDO_STACK,
DOE_DESIGNATED_FDO
Unknown flags 0x10000000
Characteristics (0x00000080) FILE_AUTOGENERATED_DEVICE_NAME
AttachedDevice (Upper) ffffbd8792fcd060 \Driver\mshidkmdf
AttachedTo (Lower) ffffbd878ccecd30 \Driver\PnpManager
Device queue is not busy.