Server 2016 BSOD Bugcheck

Availability for the Always-On Enterprise

Re: Server 2016 BSOD Bugcheck

Veeam Logoby Delo123 » Fri Mar 03, 2017 9:27 am

Ah... you see in the KB they tell you to disable interrupt remapping in ESX, thus disabling it good and for all in the bios should be good also. What actually happens is that most pci-express devices (hba's, nics etc...) will stop responding thus crashing the server. This issue has however been known since 2010 orso, so i wonder why it starts popping op now again
Delo123
Expert
 
Posts: 351
Liked: 101 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Mon Mar 06, 2017 2:43 pm

Day 4 and no daily crashes.
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby lepphce1 » Mon Mar 06, 2017 4:02 pm 1 person likes this post

@kubimike Thanks for the instructions, I've enabled the verifier so we will see what happens. I am doing a minor update to the idrac-lifecycle controller right now (sounds the same as what you describe on the HP side), we will see if it helps (which I kind of doubt). I will also check on the interrupt remapping.
lepphce1
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jun 28, 2016 4:40 pm

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Mon Mar 06, 2017 4:42 pm

Right on :)
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby Gostev » Mon Mar 06, 2017 6:41 pm

I've seen Verifier "fixing" issues before by preventing deadlocks via simply slowing things down. For example, we could never reproduce that nasty Windows Server 2012 data corruption issue I like to talk about at conferences under Verifier, which is why it is still out there (and I wonder if Windows Server 2016 has inherited it too).
Gostev
Veeam Software
 
Posts: 21510
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Mon Mar 06, 2017 9:45 pm 1 person likes this post

Great so, I'll just run verifier forever lol. I see if my server came equipped with the "Turbo" button from back in the day. Maybe I can drop it down to 12mHZ
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby lepphce1 » Wed Mar 15, 2017 8:45 pm

Is there any sense that this issue might be ReFS related? I'm thinking about installing the March update, turning off verifier and seeing if the issue persists.
lepphce1
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jun 28, 2016 4:40 pm

Re: Server 2016 BSOD Bugcheck

Veeam Logoby Gostev » Wed Mar 15, 2017 9:07 pm

It is very unlikely the issue is related to ReFS, as way too many people are using ReFS for their backup repositories by now.
Gostev
Veeam Software
 
Posts: 21510
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Wed Mar 15, 2017 9:08 pm

Really? I think this is an ReFS issue. I read your post wrong so Im reposting. Yes I was going to try just patching it but its stable with verifier turned on. Too scared to patch!
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby lepphce1 » Wed Mar 15, 2017 9:37 pm

Verifier is really slowing down my system, and the BSOD is happening sporadically enough that I figure it's worth a shot. I can always re-enable it. Who knows maybe MS fixed something else?? I'll report back on my findings.
lepphce1
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jun 28, 2016 4:40 pm

Re: Server 2016 BSOD Bugcheck

Veeam Logoby lepphce1 » Thu Mar 16, 2017 1:34 pm

@kubimike, my server bombed last night while trying to install the patch. Even though I turned off verifier I had not yet rebooted so it was still running. It looks like there is additional information in the dmp file. What do you need from me to submit to Microsoft? We can sort it out via PM. Thanks!
lepphce1
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jun 28, 2016 4:40 pm

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Thu Mar 16, 2017 3:01 pm

@leppche1 crap, this is why I didn't want to install the patch christ! PM SENT! Anything that we make from this I'll post in the forums so everyone can see.
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby lepphce1 » Thu Mar 16, 2017 5:29 pm

@kubimike, I sent you a link to the file on PM. Unfortunately I didn't realize that I was a couple of months behind on patching this thing, so I *hope* we are seeing the same issue to resolve it. My patch management solution sometimes has a mind of it's own...
lepphce1
Enthusiast
 
Posts: 30
Liked: 2 times
Joined: Tue Jun 28, 2016 4:40 pm

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Thu Mar 16, 2017 6:00 pm

Got some good data out of @lepphche1's machine. Seems like msiscsi.sys, is causing these BSODs for him. I was using iSCSI for awhile and since haven't reconfigured it when I reinstalled the OS (Just using HOTADD mode). That probably would explain why I am seeing less crashes as well since its not in use. His crash code with verifier turned on is 0xD5. I've submitted this to Microsoft under my ticket #


Code: Select all
FAILURE_BUCKET_ID:  0xD5_VRF_R_INVALID_msiscsi!iSpProcessIScsiTaskTimeout

BUCKET_ID:  0xD5_VRF_R_INVALID_msiscsi!iSpProcessIScsiTaskTimeout

PRIMARY_PROBLEM_CLASS:  0xD5_VRF_R_INVALID_msiscsi!iSpProcessIScsiTaskTimeout
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

Re: Server 2016 BSOD Bugcheck

Veeam Logoby kubimike » Thu Mar 16, 2017 6:06 pm

@lepphche I just have KBKB3211320 installed at the moment. Aside from SQL patches and Windows Defender what else do you have ? Drop into powershell issue a get-hotfix see whats up
kubimike
Expert
 
Posts: 243
Liked: 23 times
Joined: Fri Feb 03, 2017 2:34 pm
Full Name: MikeO

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 43 guests