Windows 2012 R2 DC fails to boot in Sure Backup

VMware specific discussions

Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Tue Sep 30, 2014 3:44 pm

I have a case open for this: 00644604

I have an issue with a Sure Backup job. My Windows 2012 R2 DC suddenly fails to come up after working fine every day for many months. It fails where it does the DNS, DC and GC tests. The error code is 10061. The support engineer suggested I check our production version of this VM to see if these services are working there. That's not very helpful. He also suggested I delay the VMware Tools services at startup, which I will try tomorrow after tonight's backups. Here's what I've done so far:

1. Booted up the single DC in a SB job with no other vms. It fails the same way.
2. Booted up the single DC in a SB job and watched the console. When it is up I try to login to it, but it says there are no login servers available.
3. Booted up the problem DC with another DC (Windows 2008 R2). Set the other DC to boot first, then when the problem DC booted up I was able to log into it. Here I could see that the problem DC is booted up in Safe Mode and never leaves there. That explains why the DNS test doesn't pass as the DNS service doesn't start in Safe Mode.

I noticed that this started happening after I installed a few month's worth of MS patches on the problem DC. It didn't happen the very night I installed, them, but it did start happening the next day and has been a problem since. With that, I'm not sure if one of the MS patches could be the issue. Any help or suggestions I can get on here would be appreciated.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby foggy » Thu Oct 02, 2014 11:59 am

Looks like the DC stays in DSRM mode longer than usual. You can try to increase application group timeouts and see whether this helps.

Also, the list of installed MS updates could help support to identify the issue.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Thu Oct 02, 2014 1:40 pm

I'm not getting anywhere with support. Their suggestions are not helpful at all I don't feel the tech has a grasp on what is going on. I've tried extending the time to 10 minutes and that doesn't help. When I watch the console, it boots up into safe mode and then never leaves that state. It just sits there until 10 minutes passes and the lab fails. If I login to the console and manually run the commands that I think is supposed to happen automatically, and it reboots all is fine. But then the next time I run it again it fails unless I manually do this.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby foggy » Thu Oct 02, 2014 2:37 pm

As far as I can get from the case notes, engineer's suggestions could help identify the nature of the issue. However, if you feel your case is not handled properly, you can always ask to escalate it to a higher tier.

10 minutes could still be not enough. Setting extremely high timeout (e.g. 1 hour) can let it boot finally. Probably MS could clarify why does it take so long to go through DSRM.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Thu Oct 02, 2014 3:50 pm

In one response he asks me if I have the AD server checks set on the SB job. That was after I told him initially that it fails at that point, so obviously they are set.

Then he has me run the commands manually that VEEAM should do for me while the VM is in safe mode, which is great, but then as soon as the SB lab is shutdown that is all gone and the problem will happen every time after. He thought this was a resolution, not a test.

We have been a VEEAM customer for several years now and I dread calling into support. I like that they are US based, but I never seem to get techs who are able to help me. The one time I did get good assistance was when I ended up getting escalated, but that took many days to get to. That's when I end up figuring things out on my own or by coming on here and getting the input of you guys and other customers.

I will set the timeout for one hour and see what happens and post back. I can say this isn't an issue with Windows 2008 R2 DCs in our environment.

Thanks for the quick response.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Thu Oct 02, 2014 5:21 pm

After letting it go one full hour it still times out with the application group failing to start.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby VladV » Thu Oct 02, 2014 8:36 pm

Try running the commands listed under Solution from this article manually and see if it reboots in normal mode.

http://www.veeam.com/kb1277

Maybe this will help the techs figure out what is happening.

I had this issue with 2008R2 (case #00459425) and, like in your case, it took a lot of remote sessions and a lot of emails just to make the support team understand the situation (although it should have been clear from the first remote session). After approximately 2 months (if I remember correctly) of emails and support sessions they figured out the cause and sent me a patch before the final bugfix was released in Patch #4 (build 7.0.0.871).
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Fri Oct 03, 2014 1:50 pm

Hi Vlad,

I did try the commands in the article above. I booted up the DC in SureBackup, then while it was waiting the 10 minutes that I had set for the application initialization, I logged into the console and entered these commands and rebooted. Then the server came up like it should. I guess for some reason this isn't happening automatically. However, once I do this it's just for the current SB job and the next time it runs it's broken just like before.

I feel your pain with support. They are friendly and nice to work with, but I feel like it takes way too long to get a resolution and I find it faster to come on here and seek help.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Fri Oct 03, 2014 1:53 pm

And also, I do have Patch 4 on.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Mon Oct 06, 2014 5:33 pm

I'm asking for escalation with support and getting them new log files.

My other issue with support is that they tend to send you one email per day and then don't communicate back until the next day. That makes resolution take forever.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby VladV » Mon Oct 06, 2014 7:19 pm

Yes, that happens to me every time (except if you have a critical case). Because of this you have to be extremely careful when writing. If not, after 24 hours, you find out that support misunderstood your email and you have to wait another day to see if you were clear enough the second time around.
VladV
Expert
 
Posts: 214
Liked: 24 times
Joined: Tue Apr 30, 2013 7:38 am
Full Name: Vlad Valeriu Velciu

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Wed Oct 08, 2014 7:46 pm 1 person likes this post

After a week of working with the first tech that took my case, I asked to be escalated. Once escalated, the tech I then got was able to figure out the issue within 5 minutes. It's a bug that requires a dll replacement within the VEEAM application. The fix file was named Fix_30998, so I assume the number 30998 is a number associated with this bug/issue. I asked how the first tech wasn't able to help me with this after spending a week on it, and the second level tech mentioned that this issue is listed in their internal wiki page under their bug fixes, and he would remind that initial tech to check that wiki for things like this.

Hopefully that helps others who might run into the same issue. I'm not sure if this is the same fix that you had Vlad, but if so, it must not be included in patch 4.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby foggy » Thu Oct 09, 2014 11:37 am

cffit wrote:After a week of working with the first tech that took my case, I asked to be escalated. Once escalated, the tech I then got was able to figure out the issue within 5 minutes.

That's why I've actually encouraged you to escalate the case a week ago. ;)

cffit wrote:Hopefully that helps others who might run into the same issue. I'm not sure if this is the same fix that you had Vlad, but if so, it must not be included in patch 4.

I've just checked and this particular fix was not included in patch 4, but will be addressed in one of the following releases/updates. Thanks for getting back with the update!
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby cffit » Wed Nov 12, 2014 4:10 pm

This broke again after upgrading to v8. I have a case open again on it. Just wanted people to know who might be reading this. I thought the issue was fixed in v8, but it's not looking that way.
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Windows 2012 R2 DC fails to boot in Sure Backup

Veeam Logoby foggy » Wed Nov 12, 2014 4:16 pm

No, not in v8. However, a private fix will be made available for v8 as well.
foggy
Veeam Software
 
Posts: 14752
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Next

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 19 guests