Veeam B&R recovery of a domain controller

[emytjls]

Hi
Trying to find some documentation on how to do this with Veeam, or do I still need to be taking system state backups of my DCs? For example let's say I deleted something from AD that I can't recover with Veeam Explorer for AD e.g. I've deleted an AD integrated DNS zone and I need to restore it.
Thanks for any advice.

[foggy]

Jason, please review a couple of previous pages in this thread, should answer your question. Depending on the actual scenario, you might not need an authoritative restore at all.

[emytjls]

Hi Foggy,
I've had a scan through of all 13 pages but can't find a definitive answer! Take the scenario I've given in my first post (I've deleted an AD integrated DNS zone and I need to restore it). How would I recover this?

Thanks

[foggy]

To avoid restore of the entire DC, you can read DNS records from the restored VM and recreate them in the production VM manually. Alternatively, DNS zone can be recovered from AD Recycle Bin, if it is enabled.

Otherwise, if you're after authoritative restore of the entire AD database (so that all other DCs accept your changes), you need to restore it with network disabled (to prevent default non-authoritative restore), wait until it reboots second time normally and perform authoritative restore manually.

[james411]

All,

I think I know the answer to this, but I am hoping for confirmation from the experts. Assuming a small environment with only one host and running only one DC as a guest VM, is it safe to use Veeam to backup and restore this Domain Controller? I do believe if the backup job uses Application Aware processing, then restoring the DC should be no problem and I should not run into any issues such as USN rollback is that correct? Thank you! I'd be using Veeam v8 by the way.

[foggy]

You're right, in case of a single DC environment, everything is performed automatically, provided AAIP is enabled in the backup job.

[Tos]

Customer have question about Full Restore of Active Directory.

They have two AD Servers.
If they run Full Restore of child domain AD Server, they're afraid that Veeam performs Authoritative Restore.
(If performing Authoritative Restore, this child domain AD Server is restored as parent domain.)
https://technet.microsoft.com/en-us/lib ... s.10).aspx

How Veeam do Full Restore of AD ?
If you can, let me know document about behavior of Full Restore of AD.

[Shestakov]

Toshihiro,
The behavior is explained in the last pages of the topic. Please review and ask additional questions if you have any.
Note that we may not disclose some details due to preservation of intellectual property. Thanks!

[Tos]

Hi Shestakov

I review and understand that Veeam always perform non-authoritative full AD VM restore.
Then, they have three question.

##1
When Veeam restore AD, restored AD Server restart a few times.
Do you have the index value how long it takes time until they can log in AD Server since powering on ?

##2
Don't Veeam consider that AD server completely corrupt ?
(In the case that Veeam uses non-Authoritative Restore,
they need to consider whether it needs configure physical Domain Controller Server with FSMO role.)

##3
The following document is their environment.

-Domein Contolloler#A (with FSMO's roles)
-Domein Contolloler#B
-OS : Windows Server 2008 and 2012
-Sysvol transfer : FRS or DFS

The following URL is about FRS and DFS.
https://technet.microsoft.com/ja-jp/lib ... 10%29.aspx
When Windows 2003 domain environment is upgraded to Windows 2008 domain environment, FRS will be selected if SYSVOL is not changed explicitly.
Depending on the user environment, some user doesn't change SYSVOL
even when domain environment is Windows 2008 or later.

Then, please let me know how to do the following case.

"Sysvol transfer is DFS"
Pattern 1 DC#A restore
Do they just need to Full Restore ?
Or, do they need to move FSMO's roles to DC#B ?

Pattern 2 DC#B restore
Do they just need to Full Restore ?

Pattern 3 DC#A and DC#B restore (DC#A and DC#B completely destroyed)

"Sysvol transfer is FRS"
Pattern 1 DC#A restore
Do they just need to Full Restore ?
Or, do they need to move FSMO's roles to DC#B ?

Pattern 2 DC#B restore
Do they just need to Full Restore ?

Pattern 3 DC#A and DC#B restore (DC#A and DC#B completely destroyed)
Do they just need to DC#B Full Restore after they did DC#A Full Restore and it booted ?

Regards
Toshi

[Shestakov]

Hello Toshi,
Sorry for the late reply.

##1 When Veeam restore AD, restored AD Server restart a few times.
Do you have the index value how long it takes time until they can log in AD Server since powering on ?

It`s a normal behavior that the server restarts several times.
There is no such a time index, it can vary from 2 to 30 minutes. Unfortunately Microsoft doesn`t document the processes happening inside.

##2 Don't Veeam consider that AD server completely corrupt ?
(In the case that Veeam uses non-Authoritative Restore,
they need to consider whether it needs configure physical Domain Controller Server with FSMO role.)

It`s not necessary to have a physical DC. For reliability purposes it`s recommended to keep DCs on different servers and datastores.
What kind of corruption are we talking about?
In a case of corruption, backup job will inform you and make a backup retry.

The best way to be sure of backup recoverability is a Surebackup option.

I don`t really get the 3rd question, could you clarify/rephrase it please?

Thanks!

[tscott]

I have a customer who replicates 4 VMs to an offsite host.. Their DR plan is for nightly replicas and if something catastrophic happens in their production site they will get their offsite DR host and take it onsite and power up all replicas to be back online..

The question they have is regarding active directory.. 3 of the 4 VMs they replicate are DC's.. They don't all finish at the same time (the replication jobs)..

What would happen if they powered all 4 replicas up in a DR scenario? Would AD be working or could their be issues say if 1 replica happened to fail the night before or if they finished at different times..

Thank

[foggy]

Tom, please review this thread regarding recovery of domain controllers, specifically this post should answer your questions.

[tscott]

I've reviewed it all but it's still quite confusing..

Customer is running 3 Windows 2003 DC's that are being replicated to DR site..

If production servers go kaboom and they take the DR host to production site, what are the steps?

Power on host..
Launch Veeam DR Server which has replica jobs..
Choose "Failover to replica" for the 4 replicated VMs (3 of which are 2003 DCs).
Since they are 2003 at least one of them will need authoritative restore via burflags?

Is this correct?

[tscott]

I've reviewed it all but it's still quite confusing..

Customer is running 3 Windows 2003 DC's that are being replicated to DR site..

If production servers go kaboom and they take the DR host to production site, what are the steps?

Power on host..
Launch Veeam DR Server which has replica jobs..
Choose "Failover to replica" for the 4 replicated VMs (3 of which are 2003 DCs).
Since they are 2003 at least one of them will need authoritative restore via burflags?

Is this correct?

So no one from Veeam can confirm this process? That's disheartening..

What happened to the guide that was being created 2 years ago?

Thanks

[zoltank]

I frequently do through this when restoring onto my lab host for testing. When performing a from-scratch restore I always set SYSVOL to be authoritative on the first server, however I do NOT perform an authoritative restore on the DC itself. Usually I don't even bother setting the initial sync to 0 either, as once all the DCs are up and running they'll find each other and sort themselves out.

However, please be aware that this is NOT Microsoft's recommended procedure when doing a site-wide recovery. Microsoft wants you to only restore one DC, clean up the metadata for all other DCs, set the SYSVOL to authoritative, and then promote out servers to replace the DCs. This is to prevent any potential AD issues from manifesting in the restored AD. Obviously, this isn't an issue when your server room burned down and you're going to your DR replicas, but it might be something to keep in the back of your mind.

[foggy]

Correct, in case of losing all the DC's, you'd need to force one of them to become authoritative for SYSVOL upon restore, then other DCs should recover automatically.

[Tomsyr]

Is there good documentation regarding the recommended method for Authoritative Restore of a DC in a production environment?
Thanks,
Tom

[Shestakov]

Tom, please review a couple of previous pages in this thread. Depending on the actual scenario, you might not need an authoritative restore at all. Ask additional questions if you have any. Thanks!

[vagou15]

Hi,

I have an Active directory with 2 Dc in 2012 R2 system:

1 is a virual with the fsmo roles and backuped with veeam
the second is a physical one and was just saving with classic saving tool from microsoft.

My question is: if my virtual drive with fsmo roles is broke, what is the best practises to restore it?
I think that à full restore from a snapshot is not the best way.

Before i take veeam, i restoring my ad with authoritative restore, and i read a post here http://forums.veeam.com/veeam-backup-re ... t1284.html that specify that in 2009, the autoritative restore is not supported.
So, it is supported 7 years later?
Must i do an additional backup with the microsoft tool to be abble to do an authoritative restore?
Thank you for your help.

[Gostev]

Team, let's create a support KB on authoritative AD restores with Veeam (step by step). Please coordinate with QC and Support. Thanks!

[Unison]

Team, let's create a support KB on authoritative AD restores with Veeam (step by step). Please coordinate with QC and Support. Thanks!

Cant believe this still doesnt exist
Anyone doing a full AD restore.....this chain of posts holds the valuable information they need.......just good luck to reading all 50 million posts. Every now and then someone runs into the same issue and that starts the chain off all over again (because they cant be bothered to read all the past posts).

[Outerheaven11]

I've read several documents and FAQs and nothing specifically mentions this scenario. I have a Windows Server 2012 VM acting as a DC. Can I successfully backup and restore this machine (and AD) using Veeam Free edition? Will I be able to restore my AD environment if I use an authoritative restore?

[PTide]

Hi and welcome to the community!

Can I successfully backup and restore this machine (and AD) using Veeam Free edition?

Yes, you can. You can even use Veeam AD Explorer for granular recovery.

Will I be able to restore my AD environment if I use an authoritative restore?

Yes, however it requires some manual work. Please review the thread for more info on AD restores.

Thank you.

[Gostev]

Cant believe this still doesnt exist
Anyone doing a full AD restore.....this chain of posts holds the valuable information they need.......just good luck to reading all 50 million posts. Every now and then someone runs into the same issue and that starts the chain off all over again (because they cant be bothered to read all the past posts).

Well, the only metric that triggers support KB creation is the amount of support tickets requesting this information. So, the only reason why this KB does not exist today is too few support cases requesting information on authoritative AD restore.

[Unison]

I understand that. we all do.
Massive disasters though are pretty rare.....the kind of disaster that might require this kind of restore.
With the type of disasters demanding this kind or restore being so low and not many IT Pros out there actually 'TESTING' this kind of scenario.....its understandable that 'this' is NOT one of the most frequent requests coming through to support.
But it does happen. It is a real 'thing'.....and more will continue to experience/test it........which will likely land them here (because its pretty much the only resource on it).

We all love Veeam, you guys do so much great work, build great products - so we rely on you and the team. If you dont think putting together a guide/information regarding this kind of scenario is worth the effort, not going to be used by all of your 4.32 billion customers (surely you guys are that high with adoption now ;P) then this resource right here will have to do.

But you, the veeam product, veeam support will be called on time and time again regarding this - because we choose VEEAM to backup/restore our worlds.

Not everyone is going to run across this, but it is one of those things that could be pretty easily 'fixed'....something that veeam could do for their flock.....and it would prevent that 'brick wall' feeling that most will get as soon as they hit this information resource.

[Warp7]

Hi all,

I'm having trouble yet, the situation :

2 Hosts with ESXi 5.5 U3 backuped up with Veeam v8.0.0.2084, Application Aware is on for all VMs

- 1 VM Windows 2012 DC with all roles
- 1 VM Windows 2012 DC that I use as a secondary

Yesterday my daily reverse Incremental backup has run, failed for all VMs, then I see that the first VM in the backup list is my PDC with all roles, the VM is running but many error in the event log (DHCP service error etc). I restart the VM and boom, can't boot, it goes everytime in Windows Restore mode (so not in AD restore mode). I can't boot this VM anymore.

What's the best way to return in a good situation? Restore the entire VM? I'm affraid of doing this because this it the PDC with all roles, for a secondary DC surely I would be more confident.

Thanks for your help...

[foggy]

I'm not sure this is necessary (an automatic non-authoritative recovery of the DC could be enough), but you can seize the roles to the live DC first.

[Warp7]

Hi,

Ok thanks I have done a "seizing" of all Roles, then Cleanning Metadata of the crashed DC, rebuild a new one et promote, everything seems to be ok.

I just want to open a ticket for the crash, maybe your support can help me to find wheter it comes from.

Sincerely

[foggy]

Sure, feel free to do that. They will be able to give you a hint after reviewing the log files.

[JamesNT]

Hello Everyone,

My apologies for digging up this old thread yet again (I appear to be the second person to have done so). However, it appears in testing we have a situation to ask about.

First, our setup:
* Dell PowerEdge VRTX four blade chassis.
* All hosts and all guests are Windows Server 2012 R2.
* Four node Hyper-V cluster.
* One VM is the VEEAM backup server. We are using standard edition.
* Separate R730 server for backup repository (file share)
* Two domain controllers that are both guests of the cluster.

We decided to test our backup. We installed hosts on a new, separate VRTX and proceeded to restore the VMs. Note that since we are restoring to a totally separate VRTX we are attempting to simulate a complete failure in that all we have left are backups. There is no existing infrastructure to connect to. Each VRTX is an island unto itself.

Upon restore, it is important to note that the cluster the domain controllers were running on no longer exists. VEEAM wants to, upon choosing to do a full VM restore, look for that cluster and restore to that. Not an option. So in order to restore the backup, we have to choose to restore the individual VHDX files and config files to a folder on one of the hosts, use Hyper-V Manager to import the DC's, setup a new cluster, then move the domain controllers into the new cluster.

The problem is that after a few minutes, the cluster became broken and the new hosts were no longer members of the domain even though we joined them. Attempting to log on to the hosts using the domain administrator credentials yielded the error from the host that the domain database did not have an account for that computer. After finally logging in to the main domain controller, the new computer accounts were indeed gone. So it appears we did have a USN rollback even though the domain controllers were backed up together, restored together, and powered back on together. DCDIAG revealed replication errors such as the machines having slow communication even though the event log gave the event that all situations preventing replication had been cleared. We were able to solve our issues by using NTDSUTIL to remove the second domain controller and keep the first one (at least the issues appear solved - we've had a new cluster on the second VRTX running for a couple of days now with no issue and was even able to bitlocker the CSV).

After reading this thread, we noted that we did NOT have Application Aware Processing turned on for the domain controllers. However, given the approach we have to take to restore the DC's, with having to do individual files and then import them using Hyper-V manager, would that make a difference for us? Or are we destined to have to always bring in one DC and remove the other every time we do a restore?

JamesNT