How to debug IO_REMOVE_LOCK ?

davidpreetham · Apr 15, 2020

Hi,
I am having system crash during HLK testing sleep in enable and disable test case. Driver verifier is crashing my driver saying access violation. Same TAG is not used for releasing which is used for acquiring. From description of bugcheck I understood driver verifier keeps its own Chk build io_remove_lock in place actual driver remove lock if the driver is not Chk build driver.

My query is I am not able to debug Chk build io_remove_lock to see tag information. Windbg not able to display tag details. I tired !remlock looks like deprecated.

typedef struct _IO_REMOVE_LOCK { IO_REMOVE_LOCK_COMMON_BLOCK Common; #if DBG
IO_REMOVE_LOCK_DBG_BLOCK Dbg;
#endif
} IO_REMOVE_LOCK, *PIO_REMOVE_LOCK;

TAG is checked only on case of Chk build or verifier is enabled..
-------_-------------------------------------------------------------------+++
DRIVER_VERIFIER_DETECTED_VIOLATION (c4)
A device driver attempting to corrupt the system has been caught. This is
because the driver was specified in the registry as being suspect (by the
administrator) and the kernel has enabled substantial checking of this driver.
If the driver attempts to corrupt the system, bugchecks 0xC4, 0xC1 and 0xA will
be among the most commonly seen crashes.
Arguments:
Arg1: 00000000000000d5, IoReleaseRemoveLock tag doesn't match previous IoAcquireRemoveLock tag.
Arg2: fffffa80178213e0, Address of the chk build Remove Lock structure.
Arg3: fffffa8017954070, Tag that doesn't match previous IoAcquireRemoveLock tag.
If the driver calling IoReleaseRemoveLock is not built chk,
Parameter 2 is the chk build Remove Lock used by the Driver Verifier
on behalf of the driver. In this case, the address of the RemoveLock
used by the driver is not used at all, because the Driver Verifier is
replacing the lock address for all the Remove Lock APIs.
Arg4: 0000000000000000

MrPepka · Apr 16, 2020

How do I understand the crash applies to the driver you wrote? What does your driver do? Send crash dumps

davidpreetham · Apr 16, 2020

MrPepka said:
How do I understand the crash applies to the driver you wrote? What does your driver do? Send crash dumps

Thanks for reply @MrPepka
I am not able to attach dump due to large size even after compressed.

code is huge so many places releasing and acquiring remove in dispatch functions passing TAG as IRP, sometimes device extensions based and sometimes NULL based on requirement, in IRP_MN_REMOVE_DEVICE IoReleaseRemoveLockAndWait is used to make sure released properly.
This issue is not happening if I disable tracing(tmh) HLK is also getting passed.
Is there any driver verifier issue since it is replacing its own checked build remove lock why issue is not reproduced if tracing is disabled.
Kindly let me know if you required any debug commands output..

It is modem USB driver, I am running HLK following test is failing DF - Sleep with IO During (Reliability)
STACK_TEXT:
ffffc881`b5be7248 fffff800`7b8256e3 : 00000000`000000c4 00000000`000000d5 ffff9d0f`92c366d0 ffff9d0f`a913c940 : nt!KeBugCheckEx
ffffc881`b5be7250 fffff800`7b82adfd : ffff9d0f`92c366d0 ffff9d0f`a913c900 ffff9d0f`92c36740 00000000`00000000 : nt!VerifierBugCheckIfAppropriate+0xdf
ffffc881`b5be7290 fffff800`7b0b1e19 : ffff9d0f`a913c940 ffff9d0f`92c366d0 ffff9d0f`92c366c0 00000000`00000000 : nt!VfRemLockReportBadReleaseTag+0x1d
ffffc881`b5be72d0 fffff800`7b82ad02 : 00000000`00000020 ffff9d0f`a913c940 ffffc881`b5be7358 ffff9d0f`a913c940 : nt!IoReleaseRemoveLockEx+0x1a2cf9
ffffc881`b5be7320 fffff800`815377f8 : ffff9d0f`a8580fd0 ffff9d0f`a3993528 00000000`00000926 ffff9d0f`a39931d8 : nt!VerifierIoReleaseRemoveLockEx+0x52
ffffc881`b5be7350 fffff800`81536c37 : ffffc881`b5be74f0 ffff9d0f`a3993528 ffff9d0f`a39931a0 ffff9d0f`a399357c : usbser!IdleNotificationIrpCompletionEpisode+0x350
ffffc881`b5be73a0 fffff800`8152736d : 01d6131b`0a36fb1c 00000000`00000000 00000000`00000000 01d6131b`09aadb68 : usbser!GetOutOfIdleState+0x7b
ffffc881`b5be73d0 fffff800`7af20cd5 : ffff9d0f`a3998000 fffff800`81526401 ffff9d0f`a3993100 00002def`bd9bbf00 : usbser!ReadInterruptPipe+0xecd
ffffc881`b5be7550 fffff800`7b07e998 : ffffcd00`9f09c180 ffff9d0f`a3998080 fffff800`7af20c80 5f5f5f5f`5f5f5f5f : nt!PspSystemThreadStartup+0x55
ffffc881`b5be75a0 00000000`00000000 : ffffc881`b5be8000 ffffc881`b5be1000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x28

MrPepka · Apr 16, 2020

In this case, send crash dumps via the cloud (Google Drive, Dropbox or OneDrive)

Search

Search

How to debug IO_REMOVE_LOCK ?

davidpreetham

Member

MrPepka

Sysnative Staff, BSOD Kernel Dump Senior Analyst

davidpreetham

Member

MrPepka

Sysnative Staff, BSOD Kernel Dump Senior Analyst