Jared
Sysnative Staff, BSOD Kernel Dump Expert
- Feb 3, 2014
- 1,591
Today I thought I'd write a bit about device stacks and driver stacks and how they implement IRPs.
I'm not going into detail on how drivers function and the types of drivers as I would be here all day so I'll save that for another time.
What is a device object and a driver object?
A device object is an opaque structure that represents a device or function. A device object is an instance of the DEVICE_OBJECT data structure which is used by the operating system to represent a device.
Some device objects don't always represent a physical device, they can represent a logical device.
A driver object is just a Kernel image to represent the Kernel mode driver which includes a pointer to the driver's routines.
When a driver initialises it creates a device object to represent physical or logical devices.
Device stacks and Device nodes
The Kernel organises drivers into a tree structure called the Plug and Play device tree containing device nodes that represent devices, do note that some nodes represent software components which don't have any physical devices attached to them.
A device stack contains a PDO (Physical Device Object) which represents the physical device connected to a physical bus on the motherboard, in this case I'll talk about the PCI bus as an example.
The PCI bus enumerates the child devices which are connected to the PCI bus on the motherboard, this creates the PDO for each device and is then represented by a device node in the PnP device tree.
Do note that depending on your perspective determines what type of driver the pci.sys driver is, for example if you're looking at the PCI bus device node then it's the function driver but if you're looking at one of the PCI bus node child devices associated with it then it's the bus driver.
After the device node has been associated with the new PDO the PnP manager then searches the registry for the driver(s) which needs to be part of the device stack, these drivers are called Function Drivers.
Here's a small point about the drivers usually found in device stacks:
This is the device stack for the cd drive in the computer which shows the associated device objects and driver objects within it.
For more information on Driver Verifier see here: Driver Verifier (Windows Drivers)
Driver Stacks are determined by how many drivers are present when processing an IRP by passing it down a device stack or in some cases multiple device stacks.
A driver object can be associated with multiple different device objects and therefore lots of device stacks, this shows that an IRP can be passed down lots of device stacks but only being serviced by a few drivers.
So as proven here we can clearly see that the ACPI.sys driver is associated with a lot of device objects as it can't just represent one device otherwise one hardware component would use ACPI and everything else would be powered on all the time, think about how many USB devices would be turned on.
So our CD drive is just one component that uses ACPI.
Finally we can see information about the IRP being sent by looking at the IRP data structure.
Some of the entries are pretty obvious from the name and some aren't documented, the ones that are can be found here:
IRP (Windows Drivers)
Power IRPs
I found an old dump file which was a 0x9F Kernel dump file caused by a power IRP not synchronising with the pnp manager.
Power IRPs are used to change the power state for a device and therefore they must reach the bottom of the device stack which is the physical device object.
So we can see our 0x9F bugcheck with a power IRP failing to synchronise with the PnP manager because the IRP hasn't reached the bottom of the stack.
We can see two locks have been held, IopDeviceTreeLock is to synchronise the device tree as a spinlock and the PiEngineLock which is a pnp and power management lock. The PiEngineLock is being owned by the ZTEusbnet driver in order to pass down the power IRP.
I'm not sure why but the ZTEusbnet driver isn't processing the power IRP, it's just leaving it and that's what caused the system to crash.
It'd be nice to know exactly why it didn't pass the power IRP on.
I'm not suprised though given the date of the driver.
We can see that it was meant to pass the power IRP down to the USB common class generic parent driver which, to put it simply exposes each USB composite device in order to seperate it to a single device. Passing it down to the USB bus driver should change the power state.
This is a dump file from quite a while ago but if memory serves me correctly I think an update solved the issue.
Any other questions feel free to ask, I believe I've covered most things without going into detail about drivers.
Sources: Device nodes and device stacks (Windows Drivers)
Driver stacks (Windows Drivers)
I'm not going into detail on how drivers function and the types of drivers as I would be here all day so I'll save that for another time.
What is a device object and a driver object?
A device object is an opaque structure that represents a device or function. A device object is an instance of the DEVICE_OBJECT data structure which is used by the operating system to represent a device.
Some device objects don't always represent a physical device, they can represent a logical device.
A driver object is just a Kernel image to represent the Kernel mode driver which includes a pointer to the driver's routines.
When a driver initialises it creates a device object to represent physical or logical devices.
Device stacks and Device nodes
The Kernel organises drivers into a tree structure called the Plug and Play device tree containing device nodes that represent devices, do note that some nodes represent software components which don't have any physical devices attached to them.
A device stack contains a PDO (Physical Device Object) which represents the physical device connected to a physical bus on the motherboard, in this case I'll talk about the PCI bus as an example.
The PCI bus enumerates the child devices which are connected to the PCI bus on the motherboard, this creates the PDO for each device and is then represented by a device node in the PnP device tree.
Do note that depending on your perspective determines what type of driver the pci.sys driver is, for example if you're looking at the PCI bus device node then it's the function driver but if you're looking at one of the PCI bus node child devices associated with it then it's the bus driver.
After the device node has been associated with the new PDO the PnP manager then searches the registry for the driver(s) which needs to be part of the device stack, these drivers are called Function Drivers.
Here's a small point about the drivers usually found in device stacks:
- Bus drivers detect and inform the PnP manager about its devices on its bus as well as controlling the power to the bus. There is only allowed to be one bus driver at once and Microsoft normally supplies them.
- Function drivers on the other hand are the main driver that represents the device and performs the basic operations for reading and writing, it's the driver that knows the most about its device.
- Filter drivers modify the device behaviour when needed and it's located above or below the function driver. It normally fixes errors that are detected before it reaches the function driver on the stack.
Code:
0: kd> [COLOR="#008000"]!devstack fffffa8004615680[/COLOR]
!DevObj !DrvObj !DevExt ObjectName
fffffa8005f8c150 [COLOR="#EE82EE"]\DRIVER\VERIFIER_FILTER[/COLOR]fffffa8005f8c2a0
fffffa8005f8c390 [COLOR="#FF0000"]*** ERROR: Module load completed but symbols could not be loaded for GEARAspiWDM.sys
\Driver\GEARAspiWDM[/COLOR]fffffa8005f8c4e0
fffffa8005f202e0 [COLOR="#EE82EE"]\DRIVER\VERIFIER_FILTER[/COLOR]fffffa8005f20430
fffffa8005eec060 [COLOR="#800080"]\Driver\cdrom[/COLOR] fffffa8005f77b80 CdRom0
fffffa80057379b0 [COLOR="#00FFFF"]\Driver\ACPI[/COLOR] fffffa80047f6a00
> fffffa8004615680 [COLOR="#0000FF"]\Driver\atapi[/COLOR] fffffa80046157d0 IdeDeviceP0T1L0-5
!DevNode fffffa8005742900 :
DeviceInst is "IDE\CdRomATAPI_iHAS124___B_______________________AL0R____\5&f437ab5&0&0.1.0"
ServiceName is "cdrom"This is the device stack for the cd drive in the computer which shows the associated device objects and driver objects within it.
- Atapi provides the interface to enable support for cd players.
- ACPI is the bus filter driver that enables Power Management for the operating system so when devices are not in use (In this case the cd player) it will be powered off.
- cdrom is the function driver for the cd drive that allows discs to be read and written to.
- GEARAspiWDM.sys is the cdrom 3rd party filter driver.
- VERIFIER_FILTER are filter drivers used by Driver Verifier which is enabled to monitor driver routines and operations to make sure everything is working correctly.
For more information on Driver Verifier see here: Driver Verifier (Windows Drivers)
Driver Stacks are determined by how many drivers are present when processing an IRP by passing it down a device stack or in some cases multiple device stacks.
A driver object can be associated with multiple different device objects and therefore lots of device stacks, this shows that an IRP can be passed down lots of device stacks but only being serviced by a few drivers.
Code:
0: kd> [COLOR="#008000"]!drvobj \Driver\ACPI[/COLOR]
Driver object (fffffa80039a6af0) is for:
\Driver\ACPI
Driver Extension List: (id , addr)
[COLOR="#800080"]Device Object list:
fffffa80057379b0 fffffa800573a9b0 fffffa80057399b0 fffffa800572fc20
fffffa800572fe40 fffffa800572ea00 fffffa800572ec20 fffffa800572ee40
fffffa800572da00 fffffa800572dc20 fffffa800572de40 fffffa8005819e40
fffffa8005814e40 fffffa800580fe40 fffffa800572a9b0 fffffa80057289b0
fffffa8005720e40 fffffa800571c920 fffffa800571cb20 fffffa800571bc40
fffffa800571be40 fffffa8005713bc0 fffffa8005700e40 fffffa80056ffa40
fffffa80056ffc40 fffffa80056ffe40 fffffa80056fea40 fffffa80056fec40
fffffa80056fee40 fffffa80056fda40 fffffa80056fdc40 fffffa80056fde40
fffffa80056fca40 fffffa80056fcc40 fffffa80056fce40 fffffa8004616330
fffffa8004616040 fffffa8004616c20 fffffa8004616e40 fffffa80047f9770
fffffa80039eadb0 fffffa8004be1060 fffffa80047fe170 fffffa80047fe390
fffffa80047fe5b0 fffffa80047fe7d0 fffffa80047fe9f0 fffffa80047fec10
fffffa80039a7040 [/COLOR]So as proven here we can clearly see that the ACPI.sys driver is associated with a lot of device objects as it can't just represent one device otherwise one hardware component would use ACPI and everything else would be powered on all the time, think about how many USB devices would be turned on.
So our CD drive is just one component that uses ACPI.
Finally we can see information about the IRP being sent by looking at the IRP data structure.
Code:
0: kd> [COLOR="#008000"]dt nt!_IRP fffff9801c458dc0[/COLOR]
+0x000 Type : 0n6
+0x002 Size : 0x238
+0x008 MdlAddress : (null)
+0x010 Flags : 0x40000000
+0x018 AssociatedIrp : <unnamed-tag>
+0x020 ThreadListEntry : _LIST_ENTRY [ 0xfffff980`1c458de0 - 0xfffff980`1c458de0 ]
+0x030 IoStatus : _IO_STATUS_BLOCK
+0x040 RequestorMode : 0 ''
+0x041 PendingReturned : 0 ''
+0x042 StackCount : 5 ''
+0x043 CurrentLocation : 1 ''
+0x044 Cancel : 0 ''
+0x045 CancelIrql : 0 ''
+0x046 ApcEnvironment : 0 ''
+0x047 AllocationFlags : 0x80 ''
+0x048 UserIosb : (null)
+0x050 UserEvent : (null)
+0x058 Overlay : <unnamed-tag>
+0x068 CancelRoutine : (null)
+0x070 UserBuffer : (null)
+0x078 Tail : <unnamed-tag>Some of the entries are pretty obvious from the name and some aren't documented, the ones that are can be found here:
IRP (Windows Drivers)
Power IRPs
I found an old dump file which was a 0x9F Kernel dump file caused by a power IRP not synchronising with the pnp manager.
Power IRPs are used to change the power state for a device and therefore they must reach the bottom of the device stack which is the physical device object.
Code:
DRIVER_POWER_STATE_FAILURE (9f)
A driver has failed to complete a power IRP within a specific time.
Arguments:
Arg1: 0000000000000004, [COLOR="#800080"][B]The power transition timed out waiting to synchronize with the Pnp
subsystem.[/B][/COLOR]
Arg2: 0000000000000258, Timeout in seconds.
Arg3: [COLOR="#008000"]fffffa8007005660[/COLOR], [B]The thread currently holding on to the Pnp lock.[/B]
Arg4: fffff800053e83d0, nt!TRIAGE_9F_PNP on Win7 and higherSo we can see our 0x9F bugcheck with a power IRP failing to synchronise with the PnP manager because the IRP hasn't reached the bottom of the stack.
Code:
0: kd> [COLOR="#008000"]!locks[/COLOR]
**** DUMP OF ALL RESOURCE OBJECTS ****
KD: Scanning for held locks..
Resource @ nt!IopDeviceTreeLock (0xfffff80003492ce0) Shared 1 owning threads
Contention Count = 1
Threads: [COLOR="#008000"]fffffa8007005660[/COLOR]-01<*>
KD: Scanning for held locks.
Resource @ nt!PiEngineLock (0xfffff80003492be0) Exclusively owned
Contention Count = 21
NumberOfExclusiveWaiters = 1
Threads: [COLOR="#008000"]fffffa8007005660[/COLOR]-01<*>
Threads Waiting On Exclusive Access:
fffffa800f308b50
KD: Scanning for held locks............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
18855 total locks, 2 locks currently heldWe can see two locks have been held, IopDeviceTreeLock is to synchronise the device tree as a spinlock and the PiEngineLock which is a pnp and power management lock. The PiEngineLock is being owned by the ZTEusbnet driver in order to pass down the power IRP.
Code:
0: kd> [COLOR="#008000"]!thread fffffa80`07005660[/COLOR]
THREAD fffffa8007005660 Cid 0004.0048 Teb: 0000000000000000 Win32Thread: 0000000000000000 WAIT: (Executive) KernelMode Non-Alertable
fffffa800d035ee8 NotificationEvent
IRP List:
[COLOR="#008000"]fffffa8008f5cc10[/COLOR]: (0006,03e8) Flags: 00000000 Mdl: 00000000
Not impersonating
DeviceMap fffff8a000008c10
Owning Process fffffa8006f8d890 Image: System
Attached Process N/A Image: N/A
Wait Start TickCount 396427 Ticks: 38463 (0:00:10:00.026)
Context Switch Count 44059 IdealProcessor: 2 NoStackSwap
UserTime 00:00:00.000
KernelTime 00:00:00.343
Win32 Start Address nt!ExpWorkerThread (0xfffff80003298150)
Stack Init fffff88003bd2c70 Current fffff88003bd2280
Base fffff88003bd3000 Limit fffff88003bcd000 Call 0
Priority 15 BasePriority 12 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
Child-SP RetAddr : Args to Child : Call Site
fffff880`03bd22c0 fffff800`032845f2 : fffffa80`07005660 fffffa80`07005660 00000000`00000000 00000000`00000000 : nt!KiSwapContext+0x7a
fffff880`03bd2400 fffff800`0329599f : fffffa80`0d0df208 fffff880`0ae9e10b fffffa80`00000000 00000000`00000000 : nt!KiCommitThreadWait+0x1d2
fffff880`03bd2490 fffff880`0ae915dd : fffffa80`0d035000 00000000`00000000 fffffa80`0dd8ca00 00000000`00000000 : nt!KeWaitForSingleObject+0x19f
fffff880`03bd2530 fffff880`0ae92627 : fffffa80`0d035000 00000000`00000000 fffffa80`0c0891a0 fffff880`03bd2670 : [COLOR="#FF0000"]ZTEusbnet+0x35dd[/COLOR]
fffff880`03bd2580 fffff880`0215d809 : fffffa80`0c0891a0 fffff880`020f0ecd fffff880`03bd2670 fffffa80`091c5550 : [COLOR="#FF0000"]ZTEusbnet+0x4627[/COLOR]
fffff880`03bd25b0 fffff880`0215d7d0 : fffffa80`091c54a0 fffffa80`0c0891a0 fffff880`03bd2670 fffffa80`08fc2ac0 : [COLOR="#800080"]ndis!NdisFDevicePnPEventNotify+0x89[/COLOR]
fffff880`03bd25e0 fffff880`0215d7d0 : fffffa80`08fc2a10 fffffa80`0c0891a0 fffffa80`091f9010 fffffa80`091f90c0 : [COLOR="#800080"]ndis!NdisFDevicePnPEventNotify+0x50[/COLOR]
fffff880`03bd2610 fffff880`0219070c : fffffa80`0c0891a0 00000000`00000000 00000000`00000000 fffffa80`0c0891a0 : [COLOR="#800080"]ndis!NdisFDevicePnPEventNotify+0x50[/COLOR]
fffff880`03bd2640 fffff880`021a1da2 : 00000000`00000000 fffffa80`08f5cc10 00000000`00000000 fffffa80`0c0891a0 : ndis! ?? ::LNCPHCLB::`string'+0xddf
fffff880`03bd26f0 fffff800`034fb121 : fffffa80`091c7060 fffffa80`0c089050 fffff880`03bd2848 fffffa80`070bfa00 : [COLOR="#800080"]ndis!ndisPnPDispatch+0x843[/COLOR]
fffff880`03bd2790 fffff800`0367b3a1 : fffffa80`070bfa00 00000000`00000000 fffffa80`0dc19990 fffff880`03bd2828 : [COLOR="#00FFFF"]nt!IopSynchronousCall+0xe1[/COLOR]
fffff880`03bd2800 fffff800`03675d78 : fffffa80`09196e00 fffffa80`070bfa00 00000000`0000030a 00000000`00000308 : [COLOR="#00FFFF"]nt!IopRemoveDevice+0x101[/COLOR]
fffff880`03bd28c0 fffff800`0367aee7 : fffffa80`0dc19990 00000000`00000000 00000000`00000003 00000000`00000136 : [COLOR="#0000FF"]nt!PnpSurpriseRemoveLockedDeviceNode+0x128[/COLOR]
fffff880`03bd2900 fffff800`0367b000 : 00000000`00000000 fffff8a0`11d1c000 fffff8a0`049330d0 fffff880`03bd2a58 : [COLOR="#0000FF"]nt!PnpDeleteLockedDeviceNode+0x37[/COLOR]
fffff880`03bd2930 fffff800`0370b97f : 00000000`00000002 00000000`00000000 fffffa80`09122010 00000000`00000000 : [COLOR="#0000FF"]nt!PnpDeleteLockedDeviceNodes+0xa0[/COLOR]
fffff880`03bd29a0 fffff800`0370c53c : fffff880`03bd2b78 fffffa80`114ab700 fffffa80`07005600 fffffa80`00000000 : [COLOR="#0000FF"]nt!PnpProcessQueryRemoveAndEject+0x6cf[/COLOR]
fffff880`03bd2ae0 fffff800`035f573e : 00000000`00000000 fffffa80`114ab7d0 fffff8a0`123a25b0 00000000`00000000 : [COLOR="#0000FF"]nt!PnpProcessTargetDeviceEvent+0x4c[/COLOR]
fffff880`03bd2b10 fffff800`03298261 : fffff800`034f9f88 fffff8a0`11d1c010 fffff800`034342d8 fffff800`034342d8 : nt! ?? ::NNGAKEGL::`string'+0x54d9b
fffff880`03bd2b70 fffff800`0352b2ea : 00000000`00000000 fffffa80`07005660 00000000`00000080 fffffa80`06f8d890 : nt!ExpWorkerThread+0x111
fffff880`03bd2c00 fffff800`0327f8e6 : fffff880`03965180 fffffa80`07005660 fffff880`0396ffc0 00000000`00000000 : nt!PspSystemThreadStartup+0x5a
fffff880`03bd2c40 00000000`00000000 : fffff880`03bd3000 fffff880`03bcd000 fffff880`03bd2410 00000000`00000000 : nt!KxStartSystemThread+0x16
Code:
0: kd> [COLOR="#008000"]!irp fffffa8008f5cc10[/COLOR]
Irp is active with 10 stacks 10 is current (= 0xfffffa8008f5cf68)
No Mdl: No System Buffer: Thread fffffa8007005660: Irp stack trace.
cmd flg cl Device File Completion-Context
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[ 0, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
>[ 1b,17] 0 0 [COLOR="#008000"]fffffa800c089050[/COLOR] 00000000 00000000-00000000
[COLOR="#FF0000"]\Driver\ZTEusbnet[/COLOR]
Args: 00000000 00000000 00000000 00000000I'm not sure why but the ZTEusbnet driver isn't processing the power IRP, it's just leaving it and that's what caused the system to crash.
It'd be nice to know exactly why it didn't pass the power IRP on.
I'm not suprised though given the date of the driver.
Code:
0: kd> [COLOR="#008000"]!devstack fffffa800c089050[/COLOR]
!DevObj !DrvObj !DevExt ObjectName
> fffffa800c089050 [COLOR="#FF0000"]\Driver\ZTEusbnet[/COLOR] fffffa800c0891a0 NDMP14
fffffa80070bfa00 [COLOR="#800080"]\Driver\usbccgp[/COLOR] fffffa80070bfb50 000000a8
!DevNode fffffa800dc19990 :
DeviceInst is "USB\VID_19D2&PID_0063&MI_04\6&200b5242&0&0004"
ServiceName is "ZTEusbnet"We can see that it was meant to pass the power IRP down to the USB common class generic parent driver which, to put it simply exposes each USB composite device in order to seperate it to a single device. Passing it down to the USB bus driver should change the power state.
Code:
0: kd> [COLOR="#008000"]lmvm ZTEusbnet[/COLOR]
start end module name
fffff880`0ae8e000 fffff880`0aebc000 ZTEusbnet (no symbols)
Loaded symbol image file: ZTEusbnet.sys
Image path: \SystemRoot\system32\DRIVERS\ZTEusbnet.sys
Image name: ZTEusbnet.sys
Timestamp: [COLOR="#FF0000"]Mon Oct 13 06:50:10 2008[/COLOR] (48F2E192)
CheckSum: 000329ED
ImageSize: 0002E000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4This is a dump file from quite a while ago but if memory serves me correctly I think an update solved the issue.
Any other questions feel free to ask, I believe I've covered most things without going into detail about drivers.
Sources: Device nodes and device stacks (Windows Drivers)
Driver stacks (Windows Drivers)
Last edited:
