Windows 8.1 (possible) LogMeIn bug causing 0x44 BSOD

Code: 
7: kd> [COLOR=#008000]dt nt!_IO_STATUS_BLOCK fffffa800ea3f910[/COLOR]
   +0x000 Status           : [COLOR=#ff0000]0n207278096[/COLOR]
   +0x000 Pointer          : 0xfffffa80`0c5ad010 Void
   +0x008 Information      : 0

Status field layout can be found below.

Code: 
7: kd> [COLOR=#008000]? 0n207278096[/COLOR]
Evaluate expression: 207278096 = 00000000`[COLOR=#ff0000]0c5ad010[/COLOR]

I've evaluated the value to hexiadecimal value, to try and decipher what it could mean. It seems that it may have returned NT_ERROR. I managed to find this within the Ntdef.h Header file, I apologise if anyone has difficulty reading my dark theme for Visual Studio. You can have Private vendor defined error codes.

View attachment 6865

Code: 
7: kd> [COLOR=#008000].formats 0n207278096[/COLOR]
Evaluate expression:
  Hex:     00000000`0c5ad010
  Decimal: 207278096
  Octal:   0000000000001426550020
  Binary:  00000000 00000000 00000000 00000000 00001100 01011010 11010000 00010000 <-- 0 starts here
  Chars:   .....Z..
  Time:    Tue Jul 27 02:14:56 1976
  Float:   low 1.68567e-031 high 0
  Double:  1.02409e-315

Additional Information from the WDK -

When testing a return value, you should use one of the following system-supplied macros (defined in Ntdef.h):

NT_SUCCESS(Status) Evaluates to TRUE if the return value specified by Status is a success type (0 − 0x3FFFFFFF) or an informational type (0x40000000 − 0x7FFFFFFF).

NT_INFORMATION(Status) Evaluates to TRUE if the return value specified by Status is an informational type (0x40000000 − 0x7FFFFFFF).

NT_WARNING(Status) Evaluates to TRUE if the return value specified by Status is a warning type (0x80000000 − 0xBFFFFFFF).

NT_ERROR(Status) Evaluates to TRUE if the return value specified by Status is an error type (0xC0000000 - 0xFFFFFFFF).
Click to expand...

Code: 
7: kd> [COLOR=#008000]!irp fffffa800ea3f910 1[/COLOR]
Irp is active with 1 stacks 3 is current (= 0xfffffa800c440c10)
 No Mdl: No System Buffer: Thread [COLOR=#ff8c00]fffffa800c716900[/COLOR]:  [COLOR=#ff0000]Irp is completed.  Pending has been returned[/COLOR]
Flags = 00060900
ThreadListEntry.Flink = fffffa800ea3f930
ThreadListEntry.Blink = fffffa800ea3f930
IoStatus.Status = 00000000
IoStatus.Information = 00000118
RequestorMode = 00000001
Cancel = 00
CancelIrql = 0
ApcEnvironment = 00
UserIosb = 01b12c30
UserEvent = fffffa8009017860
Overlay.AsynchronousParameters.UserApcRoutine = 00000000
Overlay.AsynchronousParameters.UserApcContext = 01b12c30
Overlay.AllocationSize = 00000000 - 00000000
CancelRoutine = fffff8801a80188c   Hamdrv
UserBuffer = 00000000
&Tail.Overlay.DeviceQueueEntry = fffffa800ea3f988
Tail.Overlay.Thread = fffffa800c716900
Tail.Overlay.AuxiliaryBuffer = fffff803d8567998
Tail.Overlay.ListEntry.Flink = 00000000
Tail.Overlay.ListEntry.Blink = 00000000
Tail.Overlay.CurrentStackLocation = fffffa800c440c10
Tail.Overlay.OriginalFileObject = 00000000
Tail.Apc = 00580012
Tail.CompletionKey = 00580012
     cmd  flg cl Device   File     Completion-Context
 [  [COLOR=#ff0000]3[/COLOR], 0]   0  0 fffffa8009ff6060 00000000 00000000-00000000    
           \Driver\hamachi
            Args: 00000000 00000000 00000000 00000000

The 3 is the Major Function Code for IRP_MJ_READ and the Minor Function Code is IRP_MN_NORMAL from my understanding.

Code: 
7: kd> [COLOR=#008000]!devstack fffffa8009ff6060[/COLOR]
  !DevObj   !DrvObj            !DevExt   ObjectName
> fffffa8009ff6060  \Driver\hamachi    fffffa8009ff61b0  HamachiTap.C6AE2999-2529-481A-A4BA-C3ED805229CC

Code: 
7: kd> [COLOR=#008000]dt nt!_IO_STACK_LOCATION 0xfffffa800c440c10[/COLOR]
   +0x000 MajorFunction    : [COLOR=#ff0000]0x5[/COLOR] ''
   +0x001 MinorFunction    : 0 ''
   +0x002 Flags            : 0xd8 ''
   +0x003 Control          : 0 ''
   +0x008 Parameters       : <unnamed-tag>
   +0x028 DeviceObject     : (null) 
   +0x030 FileObject       : (null) 
   +0x038 CompletionRoutine : (null) 
   +0x040 Context          : (null)

The data structure shows the current stack location of the IRP. Since IRPs can be handled by different Driver Objects through Driver Stacks, and therefore may be a indication of the different driver handling a different IRP.

Code: 
7: kd> [COLOR=#008000]!drvobj fffffa800968ee60[/COLOR]
Driver object (fffffa800968ee60) is for:
 \Driver\hamachi
Driver Extension List: (id , addr)
(4e4d4944 fffffa800ef88bb0)  
Device Object list:
fffffa8009ff6060  [COLOR=#ff0000]fffffa800b3f5050[/COLOR]

Code: 
7: kd> [COLOR=#008000]!devobj fffffa800b3f5050[/COLOR]
Device object (fffffa800b3f5050) is for:
 NDMP24 \Driver\hamachi DriverObject fffffa800968ee60
Current Irp 00000000 RefCount 0 Type 00000017 Flags 00002050
Dacl fffff9a100265f51 DevExt fffffa800b3f51a0 DevObjExt fffffa800b3f6688 
ExtensionFlags (0x00000800)  DOE_DEFAULT_SD_PRESENT
Characteristics (0x00000100)  FILE_DEVICE_SECURE_OPEN
AttachedTo (Lower) fffffa800e6bc7f0 \Driver\PnpManager
Device queue is not busy.

Code: 
7: kd> [COLOR=#008000]!devstack fffffa800b3f5050[/COLOR]
  !DevObj   !DrvObj            !DevExt   ObjectName
> fffffa800b3f5050  \Driver\hamachi    fffffa800b3f51a0  [COLOR=#ff0000]NDMP24[/COLOR] <-- Network Data Management Protocol?
  fffffa800e6bc7f0  \Driver\PnpManager 00000000  0000006a
!DevNode fffffa8008e37010 :
  DeviceInst is "ROOT\NET\0000"
  ServiceName is "Hamachi"

The NDMP24 may explain the network related calls in the Call Stack for the thread which was responsible for the IRP.

Code: 
7: kd>[COLOR=#008000] !thread fffffa800c716900[/COLOR]
THREAD fffffa800c716900  Cid 1c40.1da0  Teb: 000007f7fc3b4000 Win32Thread: fffff901007ffb90 RUNNING on processor 1
IRP List:
    fffffa800f678980: (0006,0118) Flags: 00060000  Mdl: 00000000 <-- Npfs
    fffffa800ea04ee0: (0006,0118) Flags: 00060000  Mdl: 00000000 <-- Npfs
    fffffa800f5f7520: (0006,01f0) Flags: 00060900  Mdl: 00000000 <-- nsiproxy
    fffffa800c52a520: (0006,01f0) Flags: 00060800  Mdl: 00000000 <-- nsiproxy
Not impersonating
DeviceMap                 fffff8a00000c280
Owning Process            fffffa800c4c8940       Image:         [COLOR=#ff0000]hamachi-2.exe[/COLOR]
Attached Process          N/A            Image:         N/A
Wait Start TickCount      864481         Ticks: 0
Context Switch Count      43034          IdealProcessor: 0             
UserTime                  00:00:00.125
KernelTime                00:00:00.140
Win32 Start Address 0x000007fd00a44aa0
Stack Init fffff8801ce29c90 Current fffff8801ce29050
Base fffff8801ce2a000 Limit fffff8801ce24000 Call 0
Priority 8 BasePriority 8 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 PagePriority 5
Child-SP          RetAddr           : Args to Child                                                           : Call Site
fffff880`1ce29438 fffff880`00b77115 : fffffa80`0cc3b840 fffffa80`093c9270 00000000`00000000 fffffa80`08531e00 : nt!ObReferenceObjectByHandle
fffff880`1ce29440 fffff880`00b590a7 : fffff880`1ce29918 00000000`00000320 00000000`00000000 fffffa80`0cb9e620 : afd!AfdEnumNetworkEvents+0x75
fffff880`1ce294e0 fffff803`d844b47a : fffffa80`0c549610 00000000`00000000 00000000`000003a8 00000000`00000114 : afd!AfdFastIoDeviceControl+0x1097
fffff880`1ce29890 fffff803`d8460a76 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!IopXxxControlFile+0x3d9
fffff880`1ce29a20 fffff803`d8075453 : fffffa80`0c716900 00000000`01f9fb18 fffff880`1ce29aa8 00000000`01f9fb40 : nt!NtDeviceIoControlFile+0x56
fffff880`1ce29a90 000007fd`03242c5a : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ fffff880`1ce29b00)
00000000`01f9fcb8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x000007fd`03242c5a

Code: 
7: kd> [COLOR=#008000]!irp fffffa800f5f7520[/COLOR]
Irp is active with 2 stacks 2 is current (= 0xfffffa800f5f7638)
 No Mdl: No System Buffer: Thread [COLOR=#ffa500]fffffa800c716900[/COLOR]:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [  0, 0]   0  0 00000000 00000000 00000000-00000000    

            Args: 00000000 00000000 00000000 00000000
>[  3, 0]   0  1 fffffa800caf9260 fffffa8008c53d10 00000000-00000000    [COLOR=#ff0000]pending[/COLOR]
           \FileSystem\Npfs
            Args: 00002000 00000000 00000000 00000000

Npfs is a Windows driver, but I'm correct, isn't the driver attempting to complete the IRP packet again? I don't think I may be correct though, if the IRP is targeted at a different Device Object.
 
Last edited:
Update:

Getting familiar again with some of the I/O documentation, targeted Device Object by the Npfs.sys could be a different part of the Driver Stack involved, if I'm not mistaken? It just seems to make some sense, that both IRPs are both marked Pending within the same thread of the same process which is the LogMeIn program.
 
Good analysis, Harry. I guess with all of this said we were correct in originally assuming this is a bug with Hamachi.

I've added an update to your findings in my blog post - BSOD Kernel Dump Analysis: LogMeIn Hamachi Windows 8 and 8.1 Bug

If you happen to run into any yourself, it may be a good idea to kindly request if the user is okay with privately providing a kernel-dump for knowledge purposes. I've only done that once so far, but if it pops up again, I will ask again. If we happen to get our hands on a few kernels, we can surely check for whether or not Npfs.sys is consistently at a PENDING status.
 
Last edited:
I may have found another kind individual to provide a kernel-dump. I'll see tomorrow when I awake!

Also, a side note, we're at the top of the Google results for 'LogMeIn Hamachi Blue Screen' :cool3:
 
Patrick said:
Good analysis, Harry. I guess with all of this said we were correct in originally assuming this is a bug with Hamachi.

I've added an update to your findings in my blog post - BSOD Kernel Dump Analysis: LogMeIn Hamachi Windows 8 and 8.1 Bug

If you happen to run into any yourself, it may be a good idea to kindly request if the user is okay with privately providing a kernel-dump for knowledge purposes. I've only done that once so far, but if it pops up again, I will ask again. If we happen to get our hands on a few kernels, we can surely check for whether or not Npfs.sys is consistently at a PENDING status.
Click to expand...

Yes that would be good idea, and the documentation for Named Pipes can be found here - Named Pipes (Windows)

I'm not sure if there is any additional information about them in the Windows Internals book, but I'll check tonight and see what else I can find.

TechNet - Named Pipes and Mailslots
 
Last edited:
WOW, I woke up to ~3 kernel-dump emails and multiple people asking if I'd like their files. I will take a look at these and then get the links to you ASAP, Harry.
 
Hello everyone,

I just came across this thread and registered at this site to see if I can contribute and maybe find a solution to this problem.

I am having the same problem (BSOD) on Win8.1, and I am also using Logmein Hamachi. I have never had McAfee, have always had Avast (Free).

However, I would like to mention these points:

(1) I have had logmein hamachi on Windows 8 (64) for a while now, and have never had this issue.
(2) I upgraded to Win8.1 (64) and still had no issue with BSOD
(3) I recently upgraded my video card to a PNY NVS 510 and upgraded my dual monitors to Dell UP2414Q (trying to get 4K on each)
(4) First occurrence (of quite a few now) of the BSOD happened *after* the card and monitors were upgraded.

So, my inclination at first was that it was related to the video drivers, etc.

Was surprised to find this thread, and mention of Hamachi caught my eye, since I do use it. My Hamachi client is 2.2.0.114

I am a newbie here, and have not carefully understood everything in this thread, but am willing to re-read and contribute any dumps, etc.

Thanks!
 
Patrick said:
We appear to have consistency!!!!!!! Check update #5 - BSOD Kernel Dump Analysis: LogMeIn Hamachi Windows 8 and 8.1 Bug

With all of this said... Harry, I suppose we now need to figure out how and why Npfs.sys keeps appearing and being marked as pending (if we can). In the worst case, I would say we have found more than enough information for LogMeIn to hopefully quickly fix this issue.
Click to expand...

The easiest answer to that question would be for LogMeIn to examine their driver code, and then perform some tests of their own. It's great we have found a very clear and strong consistency throughout the dump files. It certainly accounts for something. I find it really fun that we are beginning to find bugs in commercial products, and then hopefully helping everyone overcome this problem.

With regards to Named Pipes and Mailslots, they're on pages 1021-1026 in Windows Internals 5th Edition. I don't have the 6th Edition, so I won't know the page numbers.
 
Indeed, I believe we have done all we can do. I saw your comment on my blog, and figure it'd be easier to answer here. I sent a ticket into LogMeIn last week when we had very minimal information. I received no response. Do you think I should send another ticket, or do you know of another way to contact? Someone that provided their kernel-dump to me told me in an email after we found consistency that they are a subscriber to LogMeIn for their commercial version and they had opened a ticket for us.

timbadia said:
Hello everyone,

I just came across this thread and registered at this site to see if I can contribute and maybe find a solution to this problem.

I am having the same problem (BSOD) on Win8.1, and I am also using Logmein Hamachi. I have never had McAfee, have always had Avast (Free).

However, I would like to mention these points:

(1) I have had logmein hamachi on Windows 8 (64) for a while now, and have never had this issue.
(2) I upgraded to Win8.1 (64) and still had no issue with BSOD
(3) I recently upgraded my video card to a PNY NVS 510 and upgraded my dual monitors to Dell UP2414Q (trying to get 4K on each)
(4) First occurrence (of quite a few now) of the BSOD happened *after* the card and monitors were upgraded.

So, my inclination at first was that it was related to the video drivers, etc.

Was surprised to find this thread, and mention of Hamachi caught my eye, since I do use it. My Hamachi client is 2.2.0.114

I am a newbie here, and have not carefully understood everything in this thread, but am willing to re-read and contribute any dumps, etc.

Thanks!
Click to expand...

Hi! Welcome to Sysnative, thanks for taking the time to register and letting us know about your issue. First off, you have uninstalled LogMeIn Hamachi and the crashes have ceased, yes? Onto contributions, the best way to do so is to install LogMeIn Hamachi, set your system to generate kernel-dumps (Enabling a Kernel-Mode Dump File (Windows Debuggers)), let it crash, and then post it here for Harry and I to take a look at to check for consistency. At this point, we have seen consistency, but more people providing kernel-dumps just to further prove it only helps!

Oh, and Harry, I just remembered! I received three kernels yesterday, and only one of them was able to dump a !thread TAIL.OVERLAY.THREAD. The rest gave me this, for example:

Code: 
[FONT=&quot]1: kd> !thread fffff802ef8af540
fffff802ef8af540 is not a thread object, interpreting as stack value...
TYPE mismatch for thread object at fffff802ef8af540[/FONT]

Any explanation towards that?
 
Last edited:
Patrick said:
Indeed, I believe we have done all we can do. I saw your comment on my blog, and figure it'd be easier to answer here. I sent a ticket into LogMeIn last week when we had very minimal information. I received no response. Do you think I should send another ticket, or do you know of another way to contact? Someone that provided their kernel-dump to me told me in an email after we found consistency that they are a subscriber to LogMeIn for their commercial version and they had opened a ticket for us.
Click to expand...

I'll try and find a e-mail address for them, I never use ticket support systems, they tend to have a very slow response time.
 
Great, let me know what you find. I also edited my post and added a question for you in there.
 
Patrick said:
Oh, and Harry, I just remembered! I received three kernels yesterday, and only one of them was able to dump a !thread TAIL.OVERLAY.THREAD. The rest gave me this, for example:

Code: 
[FONT='inherit']1: kd> !thread fffff802ef8af540
fffff802ef8af540 is not a thread object, interpreting as stack value...
TYPE mismatch for thread object at fffff802ef8af540[/FONT]

Any explanation towards that?
Click to expand...

I'm not exactly sure about that error but I've received it myself too. You could try using the !object extension on the address and then checking the Type field. This will give some information about the actual type of object.

I've started a thread on their Community forum - Stop 0x44 BSOD - Analysis Report - LogMeIn Community
 
Patrick said:
Hi! Welcome to Sysnative, thanks for taking the time to register and letting us know about your issue. First off, you have uninstalled LogMeIn Hamachi and the crashes have ceased, yes? Onto contributions, the best way to do so is to install LogMeIn Hamachi, set your system to generate kernel-dumps (Enabling a Kernel-Mode Dump File (Windows Debuggers)), let it crash, and then post it here for Harry and I to take a look at to check for consistency. At this point, we have seen consistency, but more people providing kernel-dumps just to further prove it only helps!
Click to expand...

I have *not* yet uninstalled Hamachi (currently I need it constantly since most of my work is on remote servers accessible only via a Hamachi gateway network), and am living with the BSOD. Only one more occurrence since my original post. :/

I do have a zip, am attaching it here.

If it helps, I am also on a paid Hamachi subscription, not a free one.... so in case we need volume in our voice to logmein you can count me in

I will report back once I can actually uninstall Hamachi and see if the BSOD crashes cease.

Thanks again.
 

Attachments

timbadia said:
Patrick said:
Hi! Welcome to Sysnative, thanks for taking the time to register and letting us know about your issue. First off, you have uninstalled LogMeIn Hamachi and the crashes have ceased, yes? Onto contributions, the best way to do so is to install LogMeIn Hamachi, set your system to generate kernel-dumps (Enabling a Kernel-Mode Dump File (Windows Debuggers)), let it crash, and then post it here for Harry and I to take a look at to check for consistency. At this point, we have seen consistency, but more people providing kernel-dumps just to further prove it only helps!
Click to expand...

I have *not* yet uninstalled Hamachi (currently I need it constantly since most of my work is on remote servers accessible only via a Hamachi gateway network), and am living with the BSOD. Only one more occurrence since my original post. :/

I do have a zip, am attaching it here.

If it helps, I am also on a paid Hamachi subscription, not a free one.... so in case we need volume in our voice to logmein you can count me in

I will report back once I can actually uninstall Hamachi and see if the BSOD crashes cease.

Thanks again.
Click to expand...

Thanks, I'll download the zip now. As you may have already seen, unfortunately, the only solution we have found so far is the complete removal of the program.
 
timbadia said:
Patrick said:
Hi! Welcome to Sysnative, thanks for taking the time to register and letting us know about your issue. First off, you have uninstalled LogMeIn Hamachi and the crashes have ceased, yes? Onto contributions, the best way to do so is to install LogMeIn Hamachi, set your system to generate kernel-dumps (Enabling a Kernel-Mode Dump File (Windows Debuggers)), let it crash, and then post it here for Harry and I to take a look at to check for consistency. At this point, we have seen consistency, but more people providing kernel-dumps just to further prove it only helps!
Click to expand...

I have *not* yet uninstalled Hamachi (currently I need it constantly since most of my work is on remote servers accessible only via a Hamachi gateway network), and am living with the BSOD. Only one more occurrence since my original post. :/

I do have a zip, am attaching it here.

If it helps, I am also on a paid Hamachi subscription, not a free one.... so in case we need volume in our voice to logmein you can count me in

I will report back once I can actually uninstall Hamachi and see if the BSOD crashes cease.

Thanks again.
Click to expand...

I just checked and my system settings were set to "Automatic Memory Dump" ... so the attached zip from my previous post was with that setting. Not sure if that is helpful for you or not. I have changed it to "Kernel memory dump" now.... that is better for you, correct?
 
timbadia said:
Patrick said:
Hi! Welcome to Sysnative, thanks for taking the time to register and letting us know about your issue. First off, you have uninstalled LogMeIn Hamachi and the crashes have ceased, yes? Onto contributions, the best way to do so is to install LogMeIn Hamachi, set your system to generate kernel-dumps (Enabling a Kernel-Mode Dump File (Windows Debuggers)), let it crash, and then post it here for Harry and I to take a look at to check for consistency. At this point, we have seen consistency, but more people providing kernel-dumps just to further prove it only helps!
Click to expand...

I have *not* yet uninstalled Hamachi (currently I need it constantly since most of my work is on remote servers accessible only via a Hamachi gateway network), and am living with the BSOD. Only one more occurrence since my original post. :/

I do have a zip, am attaching it here.

If it helps, I am also on a paid Hamachi subscription, not a free one.... so in case we need volume in our voice to logmein you can count me in

I will report back once I can actually uninstall Hamachi and see if the BSOD crashes cease.

Thanks again.
Click to expand...

Understood! Yes, as Harry said, we've analyzed about as much as we know we can and found a consistency enough to confidently label it a bug in the actual driver itself. Other than that, it's up to LogMeIn now to go ahead on their end and fix the bug in their driver. Since we cannot do that as we did not program this driver, all we as volunteers can do is recommend removal until LogMeIn fixes it.

Also, before you uninstall LogMeIn Hamachi, as I said, if you'd like to assist, setting a kernel-dump up and having it crash and providing that to us would be great. This is optional, but it helps us as research wise, and as I noted to further prove consistency. The .ZIP you provided contains a minidump, which although still good, we cannot unfortunately run a wide variety of extensions in to get valuable information.

Final note, which I am going to add to the top of my blog post because I need to make a huge edit to it today, I would go ahead and make a support ticket - https://secure.logmein.com/support/submitaticket/

I am going to start adding that link to every one of my 0x44 posts. Eventually LogMeIn will get so many tickets this way, they have to look into it if they are not already. I'd also like to go ahead and research whether or not there has been official word from a developer or LogMeIn employee regarding this bug.

I just checked and my system settings were set to "Automatic Memory Dump" ... so the attached zip from my previous post was with that setting. Not sure if that is helpful for you or not. I have changed it to "Kernel memory dump" now.... that is better for you, correct?
Click to expand...

Correct! When the system crashes, it will generate a MEMORY.DMP file within the directory C:\Windows

Once that happens, it'll be too large to attach here, so you'd then go ahead and upload it to a website like Skydrive, Mediafire, Google Drive, whatever works best for you and whatever can get a link to Harry & I.
 
Last edited:
My Hamachi client "updated" to version 2.2.0.130 (from what I believe was 2.2.0.114 yesterday) ..... I have not had a crash yet ... could this update have fixed it?

I must say, however, that even though my Hamachi client is installed and powered on, my VPN is down because my gateway on the gateway network is down due to a power outage at the office (due to the snow and ice there), so I will have to wait and see how my client machine behaves once that is up.

Just thought I would update you guys.

Thanks!
 
Back
Top