High Latency Issues in Windows 10 (storport.sys)

IonutZ · Jan 25, 2019

Hello friends,

First post here! Going to try my best to create a good post... I've followed a bunch of threads about reducing DPC / Measured interrupt to process latency, but to no avail.

My setup:

Desktop, age of hardware: ~2 years
Asus Z10PE-D8 WS
2x Xeon E5 2699v4
4x Samsung 32gb DDR4 2400Mhz RAM (M393A4K40BB1-CRC) (set at 2400) in slots AB / EF per mobo specs
Intel 750 SSD
Samsung 840 Pro SSD
Intel iRSTE driving 4x 10tb Hitachi HDDs in a 10 config.
Nvidia Geforce GTX 1080ti
Corsair AX1500i
2x Corsair H80i
RME HDSPe AES (Audio interface) to a Lavry DA11 over AES.

Clean UEFI install of OS Windows 10 Pro x64 Build 17763 with installer built from microsoft.com and a key purchased on ebay. The purpose of the reinstall (and migration from Windows Server 2016, installed prior, on which I believe I had some similar problems but fixed them a long time ago), was to move to a full UEFI build. I disabled CSM entirely, and am

I'm unable to answer whether "driver verified" is enabled, but I'm assuming it is. I disabled the Windows Firewall service and all Malwarebytes protection.

No proxies, 2x Intel I210 Ethernet Adapters with "Allow the computer to turn off this device to save power" disabled and "Energy Efficient Ethernet" disabled. Both cables are plugged into my switch (because usually I use port aggregation to get 2gbps out of them), but I turned one of the switch ports off and disabled port aggregation at the switch - only 1 is running.

No disk image tools. No over or underclocking. This motherboard has decent "Auto" values leading to a stable config, but I purposely set voltage on CPU and RAM to spec, BCLK to 100Mhz, and I'm letting the mobo control the multiplier and it works just fine. It's very stable, I had no problem running prime95 on 88 threads for 10 minutes and the cooling system adapts accordingly. No BSODs so far...

What prompted this post and research into latency were the audio pops. What is interesting is that when listening to music through the RME which is a PCIE card (that then connects to the Lavry DA over AES) the pops are more often than when using the Lavry DA's USB connection straight into the computer, but regardless, both pop.

LatencyMon mentioned storport.sys as being the culprit for highest DPC.

Speccy: http://speccy.piriform.com/results/x4VBQKGCzTi77vBl2t52KTm

I attached trace.etl, msinfo32.nfo, and DxDiag.txt to this post.

If anyone has any ideas on what I can try next, your advice would be greatly appreciated! Thank you :)

View attachment info.zip

IonutZ · Jan 25, 2019

Here is what LatencyMonitor looks like:

LatencyMon Stats Report
View attachment LatMon - Stats.txt

cwsink · Jan 26, 2019

Hi IonutZ - welcome to the forum :)

That driver (storport.sys) is associated with high speed disk I/O and there are some DPCs taking as long as 100 ms which is way too long. In my experience, that usually indicates a problem reading/writing a storage device. Are there any entries in Event Viewer showing disk problems? Have you checked the health of your drives?

IonutZ · Jan 26, 2019

cwsink said:
Hi IonutZ - welcome to the forum :)

That driver (storport.sys) is associated with high speed disk I/O and there are some DPCs taking as long as 100 ms which is way too long. In my experience, that usually indicates a problem reading/writing a storage device. Are there any entries in Event Viewer showing disk problems? Have you checked the health of your drives?

Thank you cwsink! Nothing notable in event viewer. Reinstalling windows right now with just the Intel 750 in there and gonna run latency monitor on a fresh install to see if there are any changes.

cwsink · Jan 26, 2019

Looking at the sequence of ISR and DPC calls happening around the time of the DPCs taking a long time it looks like the plug and play manager service gets busy. It makes me wonder if a drive is intermittently going offline and then being re-detected. The storport.sys function being called is RaidpAdapterTimerDpcRoutine which suggests to me it's a timer DPC and could be waiting on something however the more normal DPC duration appears to be in sub-millisecond range - typically less than 10 microseconds. I'm guessing it's communicating with your RAID array. I don't have much experience with diagnosing RAID problems. Does IRST have functionality to test the health of the RAID array? How old are the drives?

IonutZ · Jan 26, 2019

cwsink said:
Looking at the sequence of ISR and DPC calls happening around the time of the DPCs taking a long time it looks like the plug and play manager service gets busy. It makes me wonder if a drive is intermittently going offline and then being re-detected. The storport.sys function being called is RaidpAdapterTimerDpcRoutine which suggests to me it's a timer DPC and could be waiting on something however the more normal DPC duration appears to be in sub-millisecond range - typically less than 10 microseconds. I'm guessing it's communicating with your RAID array. I don't have much experience with diagnosing RAID problems. Does IRST have functionality to test the health of the RAID array? How old are the drives?

Must have been some bad driver. I reinstalled Windows 10 and storport.sys is no longer the problem. However I'm still experiencing high latency from a few other drivers, and highest measured interrupt to process latency is huge. Is that normal? (will come up with the rest of the diag package in the next reply - 5min)

IonutZ · Jan 26, 2019

Here are some more traces... storport.sys no longer a problem, but it seems like DirectX is still being a pain.

File is too large, uploaded @ moretraces.zip - Google Drive

cwsink · Jan 26, 2019

My understanding of LatencyMon is that the top 2 bars are actually doing a media playback simulation and should be ignored unless nothing else is running. If you're testing your own media playback the bottom 2 bars are what you should be interested in. The values from the screenshot don't suggest you should be experiencing problems. Keep in mind, if you're running LatencyMon it will be adding to your ISR and DPC processing time so your system is probably doing better than what the bottom 2 show.

The etl trace looks really good as far as ISR and DPC processing time, actually. Are you still having audio glitches? If not but the top 2 bars of LatencyMon have you worried I think it's misleading and should be ignored.

IonutZ · Jan 26, 2019

cwsink said:
My understanding of LatencyMon is that the top 2 bars are actually doing a media playback simulation and should be ignored unless nothing else is running. If you're testing your own media playback the bottom 2 bars are what you should be interested in. The values from the screenshot don't suggest you should be experiencing problems. Keep in mind, if you're running LatencyMon it will be adding to your ISR and DPC processing time so your system is probably doing better than what the bottom 2 show.

The etl trace looks really good as far as ISR and DPC processing time, actually. Are you still having audio glitches? If not but the top 2 bars of LatencyMon have you worried I think it's misleading and should be ignored.

Thank you very much for checking it out! No longer having audio glitches, just saw that those bars still indicate a very high latency and figured there's still something wrong! Again, much appreciated :) thank you sir.

cwsink · Jan 27, 2019

I'm trying to learn as much as I can about etl trace analysis so thanks for making yours available and supplying good information. As far as I can tell you should have no ISR/DPC latency problems at this point; yours is as steady as I've ever seen. Please do let us know if the problems return. Good luck!

IonutZ · Jan 27, 2019

cwsink said:
I'm trying to learn as much as I can about etl trace analysis so thanks for making yours available and supplying good information. As far as I can tell you should have no ISR/DPC latency problems at this point; yours is as steady as I've ever seen. Please do let us know if the problems return. Good luck!

I'm going to detail how I was able to achieve best results if anyone else finds this thread:

* Get latest copy of Windows 10 / Server from Microsoft and create a UEFI USB with it (not going to detail how to do that because there's plenty of tutorials on google)
* In Z10PE-D8 WS's bios, turn off CSM. That ensures that your system only recognizes UEFI compliant devices. The whole reason why I got into this world of trouble was because I wanted a full UEFI build coming from a Windows Server 2016 install on legacy that was taking forever to boot up.

* I removed all USB devices except for Keyboard / Mouse (and a couple of hubs inside of the computer that take care of the H80i pumps and the AX1500i PSU).
* Under PCH settings go to both sSATA and SATA and disable them (assuming the SSD you're installing Windows on is NVMe, if it's not, obviously don't turn off the controller for the drive you're installing Windows on, but it would be prudent to disable any other devices that are on the same controller). If you have a RAID set up on the SATA controller (that can act like an Intel RST or LSI MegaRaid) don't worry, disabling the controller will not destroy your data. Even going from RAID to AHCI by mistake shouldn't affect your data as long as you don't do anything with those HDDs. To regain your RAID config, re-enable SATA controller and choose RAID as the controller's type after the install.

* Install Windows... you really shouldn't have any problems doing this.
* Go back in BIOS and turn on your SATA and sSATA controllers (or whichever's necessary). Also be mindful of all the crap this motherboard has on by default like Serial and COM ports and other stuff you'll probably never use. It's a best practice to turn off all the things you don't think you'll ever use.
* Once in Windows, get latest Chipset and SATA/RAID drivers from asus.com - DO NOT RELY ON INTEL DRIVERS... if you read the fine print on Intel's drivers, most mobo manufacturers will CHANGE the generic Intel drivers to what they need. This was a pain in the ***! I didn't think that it would be a problem. The only way I've found Intel Rapid Storage Technology (enterprise) to work on my build was with the drivers from Asus... I think I lost a day on this, and on why storport.sys was going crazy with DPC latency ... because the drivers did not work. GET THE DRIVERS FROM ASUS.COM lol.

* After installing the Asus.com provided Intel SATA drivers (even though Windows 10 might come with newer ones) restart, at the time of writing this the latest RST driver from asus was 4.7.0.1068,

* This mobo has 2x Intel NICs (ethernet adapters). Get PROWinx64 from Intel.com. Turn off ASPM on the Ethernet Devices in Device Manager (Power Management -> Allow the computer to turn off this device to save power). Also under Advanced > Energy Efficient Ethernet -> Off
* If you have an Intel SSD like me, get the Intel SSD Toolbox.

When running latmon you should have Highest execution <5ms. Don't pay attention to the total execution, it's just a running sum.

Again, @cwsink thank you :)

Search

Search

High Latency Issues in Windows 10 (storport.sys)

IonutZ

Member

IonutZ

Member

cwsink

Sysnative Staff, BSOD Kernel Dump Expert

IonutZ

Member

cwsink

Sysnative Staff, BSOD Kernel Dump Expert

IonutZ

Member

IonutZ

Member

cwsink

Sysnative Staff, BSOD Kernel Dump Expert

IonutZ

Member

cwsink

Sysnative Staff, BSOD Kernel Dump Expert

IonutZ

Member