Comprehensive data protection for all workloads
Post Reply
khughes
Novice
Posts: 9
Liked: never
Joined: Oct 10, 2011 11:04 pm
Full Name: Kyle Hughes
Contact:

Restoring Windows 2008 Domain Controllers

Post by khughes »

We switched over to Veeam a wihle ago and have had nothing but good things with the software, the stability of the backups and restoring. This past weekend we had a D/R drill which was the first time since we made the switch to Veeam backing up our VMware environment.

While everything restored fine to the hosts, our domain was corrupted. It's being backed up via application aware so I would've thought that doing it that way would've prevented any sort of corruption issues. Should we be doing anything differently backing up the domain or should I have done anything differently restoring the domain?

I'm kind of at a loss of what went wrong as I've never had a failed D/R drill in the past 5 years until now. My initial thoughts are that the backup was just bad. Any other input would be gladly accepted.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Gostev »

What specifically do you mean by "corrupted"? Did you restore all domain controllers during your DR test, or just one of them?

One thing for sure - if the restore went fine, then backup was good (exactly identical to what was obtained from the source VM). If the backup file was corrupted, restore would fail - since part of the restore process includes verifying checksums for each restored block (any corruption would make the restore process to fail).
khughes
Novice
Posts: 9
Liked: never
Joined: Oct 10, 2011 11:04 pm
Full Name: Kyle Hughes
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by khughes »

We restored both (2 of 2) domain controllers during the DR test. What I meant by corrupted was the actual domain inside the servers was not in a functioning order. Once we restored the virtual machines, powered them on, seemed like they would talk for a few minutes then everything just basically stopped working within the domain. We weren't able to login to certain systems that didn't have our credentials cached, and weren't able to add/join systems to the domain. Tried shutting everything down, and restoring just one of them to see if we could get it up and running but that didn't work either.

Also just want to throw it out that I'm not blaming Veeam for whatever happened, just trying to get a better understanding of what may have gone wrong / why things didn't seem to work. I brought the tapes with our backup files on them back to reload those backups into my test lab to try and replicate the problem but will take a few days to get that up and running.
vMO
Expert
Posts: 176
Liked: 21 times
Joined: Jul 08, 2011 8:16 am
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by vMO »

Veeam normaly restores domain controller 'non authoritative' if you start from scratch, you have to restore first domain controller 'authoritative'
Veeam B&R v5 recovery of a domain controller
lobo519
Veteran
Posts: 315
Liked: 38 times
Joined: Sep 29, 2010 3:37 pm
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by lobo519 » 1 person likes this post

vMO wrote:Veeam normaly restores domain controller 'non authoritative' if you start from scratch, you have to restore first domain controller 'authoritative'
Veeam B&R v5 recovery of a domain controller
Instead of reading though the pages and pages on the forums - Perhaps Veeam could write a KB article on DC recovery for their knowledge base?? I think it would certainly eliminate a lot of questions here.. :wink:
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by tsightler »

Perhaps this would be a good idea, but my guess is it would be a really short paper as, in general, the process is fully automated. If you read that thread, you can see that the final outcome was pretty much "restore the DC's...wait for them to boot, reboot, sync". The final outcome was that the user didn't have to do anything except restore the VMs. This is as it should be as the process is fully automated, and any manual intervention that is require means something has gone wrong with the process. Of course, as with any automated process, things can certainly go wrong, but it's very difficult for a paper to cover all of the various things that could happen. That being said, I think having a paper for DC recovery is worth considering since it does come up on the forums on a regular basis.

For the OP, at this point it's pretty much impossible to know what might have been wrong, I can guess, but it would be just that, a guess. It appears you assumed they were "corrupt" but I think that is unlikely. My guess is that the replication services failed to start for some reason, perhaps they were not restored at the same time. Did you make any attempt to troubleshoot what was wrong with the DCs? Were you able to logon to the DCs and look at the messages in the event log? That should have given some clue. How many DCs do you have in your environment? Did you restore them all? For the ones that you restored, were they on the same subnet? Were name services working correctly in the DR environment?
lobo519
Veteran
Posts: 315
Liked: 38 times
Joined: Sep 29, 2010 3:37 pm
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by lobo519 » 1 person likes this post

tsightler wrote:Perhaps this would be a good idea, but my guess is it would be a really short paper as, in general, the process is fully automated. If you read that thread, you can see that the final outcome was pretty much "restore the DC's...wait for them to boot, reboot, sync". The final outcome was that the user didn't have to do anything except restore the VMs. This is as it should be as the process is fully automated, and any manual intervention that is require means something has gone wrong with the process. Of course, as with any automated process, things can certainly go wrong, but it's very difficult for a paper to cover all of the various things that could happen. That being said, I think having a paper for DC recovery is worth considering since it does come up on the forums on a regular basis.

For the OP, at this point it's pretty much impossible to know what might have been wrong, I can guess, but it would be just that, a guess. It appears you assumed they were "corrupt" but I think that is unlikely. My guess is that the replication services failed to start for some reason, perhaps they were not restored at the same time. Did you make any attempt to troubleshoot what was wrong with the DCs? Were you able to logon to the DCs and look at the messages in the event log? That should have given some clue. How many DCs do you have in your environment? Did you restore them all? For the ones that you restored, were they on the same subnet? Were name services working correctly in the DR environment?
I agree - I would expect it to not be much more than explaining what automatically happens when doing a restore, the normal steps that would occur, explain the fact that it is a non-authoritative restore by default and how to do an authoritative restore if needed (at at least a link to a MS KB)
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by tsightler »

OK, I'll think about how something like this might be put together and perhaps talk with out support team about the common issues they see with DC recovery.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Gostev »

tsightler wrote:My guess is that the replication services failed to start for some reason, perhaps they were not restored at the same time.
NETLOGON service stops after some time if the DC is unable to connect to any replication partner (other DC), so this very well might be the case!
kurbycar32
Influencer
Posts: 10
Liked: 2 times
Joined: Apr 04, 2012 3:27 pm
Location: Northern California
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by kurbycar32 »

I would still love to see an article that says "Yes you can backup a domain controller, here is how to do it". Popular belief is that you don't want to pull a snapshot of any type of replicated database because the restore would result in a mismatch of its partner system. Obviously Veeam makes this work by booting into restore mode but i need a document that explains this. Also setting up the job for the first time: do i want to enable application aware image processing? Should i use full backups? Please spell it out in a KB
Yuki
Veeam ProPartner
Posts: 252
Liked: 26 times
Joined: Apr 05, 2011 11:44 pm
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Yuki » 3 people like this post

I second this - while the steps and procedures may be obvious to some, it should be documented so people can find the information easier and faster. I find that searching forums can be frustrating on occasion due to certain word being used in many discussions.
kurbycar32
Influencer
Posts: 10
Liked: 2 times
Joined: Apr 04, 2012 3:27 pm
Location: Northern California
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by kurbycar32 »

Yuki wrote:it should be documented so people can find the information easier and faster.
And present said information to less technical people such as management
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Vitaliy S. »

Though it's not a whitepaper describing how to backup a DC VM, but this webinar should still answer all basic questions our new users might have > http://www.veeam.com/videos/advanced-ac ... s-107.html
kurbycar32
Influencer
Posts: 10
Liked: 2 times
Joined: Apr 04, 2012 3:27 pm
Location: Northern California
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by kurbycar32 »

That's a start but management isn't going to want to watch an hour long YouTube video. Just a simple KB article stating whats supported and how its supposed to work is all we need.
jlyle
Novice
Posts: 7
Liked: 1 time
Joined: Jan 15, 2013 9:15 pm
Full Name: Jeff Lyle
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by jlyle »

I think a KB would be a good idea! I am configuring surebackups for our DC's which pass the DC test scripts, however I configured the lab environment to remain until stopped manually as I wanted to do some manual checks (call me paranoid). Logging in and trying the AD users & computers results in domain cannot be found/contacted error message, turns out that a reboot corrects the issue but is this not going to cause an issue when I get only Exchange and the exchange services cannot start because AD needs a reboot???

The point is I agree that a KB for recommended best practices would be useful and confidence boosting for new customers, especially if common issues can be highlighted.
Yuki
Veeam ProPartner
Posts: 252
Liked: 26 times
Joined: Apr 05, 2011 11:44 pm
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Yuki » 3 people like this post

I'm not sure why this topic is taken lightly.

We have some clients on Veeam - they are public medical companies and are regulated by SOX and FDA. Many of them REQUIRE disaster recovery procedures to be fully written out in documentation so anyone can pick up the documentation package and follow the procedures to restore all systems. We don't get to debate this policy, as we don't get to debate proper user creation and termination procedures and others. So how does a KB help us with this? Simple - we can cite the source of the information when we put it together. If there are any issues with the restore by people other than the original staff who implemented the system - they can always refer to the source of the steps/procedure instead of registering on forums and searching.
lobo519
Veteran
Posts: 315
Liked: 38 times
Joined: Sep 29, 2010 3:37 pm
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by lobo519 »

I couldn't agree more.
jlyle
Novice
Posts: 7
Liked: 1 time
Joined: Jan 15, 2013 9:15 pm
Full Name: Jeff Lyle
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by jlyle »

It does look like DC backup and restore is supposed to be handled automatically, perhaps we are the minority having difficulty. I have just been escalated to teir 2 support, and am having quite a pleasant experience so far. I will post the solution to my problem when I have it on this http://forums.veeam.com/viewtopic.php?f ... 457#p70457 thread.
DamMayhem
Novice
Posts: 3
Liked: never
Joined: Apr 24, 2012 3:26 am
Full Name: Daniel Mayhew
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by DamMayhem »

Restoring Windows 2008 Domain Controllers is not recommended due to domain replication issues that will most certainly come up. However, if you are running VMware 5.1 and you upgrade your DCs to Server 2012 they will be 'VM aware' because of the new attribute 'msds-generationid'. After that, restoring (as well as cloning) them is not a problem.

Found this: http://www.sole.dk/virtualizing-your-do ... ing-fired/
technerd
Novice
Posts: 9
Liked: never
Joined: Jan 11, 2013 9:19 pm
Full Name: Antonio S.
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by technerd »

Very interesting. More curious about 2012 now. Thank you for sharing!

Very new to Veeam here, and had to comment here before moving to another thread / starting a new one.

I haven't tried a full restore of a (2008) DC yet, but can say I did run into trouble with the time service on one, which consequently caused our (Cisco) phone system to have some issues. I've since stopped using Veeam to backup our two virtual DCs, as I read elsewhere that snapshotting them can cause issues exactly like this.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Vitaliy S. »

technerd wrote:I did run into trouble with the time service on one, which consequently caused our (Cisco) phone system to have some issues. I've since stopped using Veeam to backup our two virtual DCs, as I read elsewhere that snapshotting them can cause issues exactly like this.
Issues with the time service can be caused by wrong time on the ESXi host that provides resources to a DC VM, configuring NTP service on the host does fix this issue.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by Gostev »

DamMayhem wrote:Restoring Windows 2008 Domain Controllers is not recommended due to domain replication issues that will most certainly come up.
While this might most certainly come up with other backup solutions, Veeam takes care of USN rollback issue since version 2.0. Our application-aware processing will detect if processed VM is a domain controller, and will perform the required steps during both backup and restore, resulting in correct restore with no impact on AD replication.
DamMayhem
Novice
Posts: 3
Liked: never
Joined: Apr 24, 2012 3:26 am
Full Name: Daniel Mayhew
Contact:

Re: Restoring Windows 2008 Domain Controllers

Post by DamMayhem »

That's excellent information, Gostev. I wasn't aware of that. One more reason for me to love Veeam!
Post Reply

Who is online

Users browsing this forum: ybarrap2003 and 200 guests