Multiple Domain Controllers - How to Backup?

[Unison]

Hi guys,
I have done a bit of searching around but couldnt really find a quality answer on this.
If i have multiple Domain Controllers - how should i be backing them up with veeam to ensure i can recover them both back to the exact same point in time?

If something goes wrong with one of your domain controllers and you need to restore it - then you must also be able to restore all other DCs to the exact same point in time otherwise your going to get AD corruption, UNC Roll back issues etc etc...
If you cant restore all DCs back to the same time then you may as well not even bother doing the restore.

With veeam i tried adding the two DCs to the same backup job, but it backs up one VM, and doesnt backup the second VM until the first is done.....so they will be well out of synch by then. Now i have created two separate backup jobs - one for each DC. I created the jobs a quickly as i could and both are set to backup hourly.....their 'next run time' shows only about 40 seconds in between - which is pretty close but i fear that there will still be inconsistencies between the two DCs if i ever had to restore them both back to a point in time.

Is there any way to force both the jobs to run at the exact same time? So that the snapshot is captured at the exact same point?

[Vitaliy S.]

Hi Gav,

Here is the procedure you should follow while restoring multiple DCs into a fresh environment where no other/existing DCs are available:
Active Directory and DR Site

Is there any way to force both the jobs to run at the exact same time? So that the snapshot is captured at the exact same point?

Currently it is not possible because of the reasons described in this thread: Synchronized VM's backup/replication

Hope this helps!

[zoltank]

What version of Windows are your domain controllers?

[Unison]

Thanks Vitality,
From that post i understand now that veeam is just requesting the snapshot to take place so even if i could force them to start at the exact same time, the actual snapshot might not happen at the same time.
I suspect for this case, i will just have to rely on the fact that both DCs are backed up with VSS enabled and when they are both recovered together - they will both recognise that they are 'recovered DC's' and they will 'retire' their USN identifiers and start a new set (they both did this successfully when i p2ved them with vmware converter - so hopefully they will do the same with veeam - i will test this too shortly by recovering to an isolated network.). Both are server 2003. I separated isolated each of them then p2ved them one at a time - then joined them back together in the virtual environment. The directory services log in both DCs reported knowing that the DC was 'recovered' and that the UNC identifiers of both DCs was successfully retired - using repadmin i could then see the retired UNC numbers and the new UNC number sets for both DCs and that they were successfully synching and updating with each other.
When i test the restore of both DCs to the isolated vswitch, i will test that the same thing happens. I will keep these two DCs backing up hourly together - as close as possible, because if i ever do have to recover one of them, i will choose to recover both. Would you recommend that too or is it really not necessary? Because of VSS, just the one recovered DC will 'catch up' to the good DC because it will copy back all the changes using the different UNC numbers - this would work for just a DC but not a DC that holds all the roles and GC?

Zoltank - both DCs are 2003 - do you have something to add?

Thanks guys

[Gostev]

Actually, it is almost the opposite.

Single failed DC:
Just do normal full VM restore from earlier backup. This would result in automated non-authoritative restore, and the restored DC will sync up with the other DCs automatically. There is huge topic about this around here with more details on this scenario.

Whole Active Directory restore:
1. First, perform authoritative restore of a DC from backup before corruption occurred. This is a manual process (look for an existing topic), and it will push the directory state from the restored DC to all other live DCs
2. Then, if needed also restore all non-functioning DCs normally (from earlier backups than the one that was used for authoritative restore).

I cannot vouch for the whole directory recovery process, as it really quite beyond our product and requires really good knowledge of AD (which I once had in the previous life, but forgot most things by now). But I can vouch for the single failed DC restore scenario (as this is most common restore scenario, and is exactly what you will need to do in 99.9% of cases).

[Unison]

Thanks Anton,
I will change my restore strategy for DCs.
No need to be backing them up as close together as possible. With the more likely single DC failure, i can just recover that one from a past point in time - AD will then catch it up using the still running and good DC on the network.
In the event both DCs are toast or AD gets corrupted - I will just recover one DC to just before the issue occurred....and recover the second DC to a point before the recovery time of the first recovered DC (putting it behind the first recovered DC but it will then catch up).

Thanks for the clarity Anton

[aeccles]

I was going over a restore scenario and started to wonder if backing up my Domain Controllers at different times during the night (currently 4 hours between jobs) could result in issues if I ever needed to do a full network restore. For Example if I backup DC1 at 6pm and 2 account changes are done at 7pm and then DC2 backs up at 10pm.

When I restore, DC1 could end up running for a while before DC2 is completely restored (Has an archive file store in my case of 500gb). If changes were made to my restored DC1 before DC2 is online, could they end up in an inconsistent state?

And if so, can I remedy this by putting them in the same job?

Thanks

[veremin]

You have been merged to the existing thread regarding similar issue; so, kindly take a look at the answers provided above.

As to your questions, there is no particular need to backup both DCs at the same time. In case of full AD restore, just use the backup that occurred closer to the time when an issue appeared and sync the second DC with it later.

Thanks.

[aeccles]

Thanks Eremin, That does answer my specific question. I have a related follow-up however.

After reading all of the related posts, I am a bit confused as to what the agreed upon method (Authoritative vs. Non-Authoritative) is for DC recovery in a disaster scenario in which all DCs(Server2008) must be fully recovered into a new enviornment.

From what I understand, the best method is to:
1) Restore the most recently backed up DC first in the default non-authoritative mode (as you mentioned.)
2) Wait 20-30 minutes for the process to complete
3) Perform a authoritative SYSVOL restore - http://msdn.microsoft.com/en-us/library/cc507518 (VS.85)
4) Restore the other DC, again in the standard non-authoritative mode

Does that sound correct?

[veremin]

Yep, you’ve got the point right.

Restore the most recently backed up DC first in the default non-authoritative mode (as you mentioned.)

The process of restoring domain controller in non-authoritative mode will handled for you by Veeam Backup and Replication.

Thanks.

[HussainMahfood]

I know that Veeam it has its own way to the restore of the DCs. but is there any sequence or KB to the way that we need to follow for restoring two DCs.

We have 2 DCs one running 2012 and one running 2008 r2 the forest running in 2008r2 mode. The PDC is 2008 r2 when we restore both to the test environment that is separated from production. nothing works...

*DC with 2008 r2 run in normal mode but DC services are not functional. while it is running if I restarted the Active Directory domain services service. it run for awhile like 5 to 10 minutes then stopped again.

*DC with 2012 run all the time in safe mode and using the commands mentioned in the

note: I use application aware during the backup

Veeam it should have a clear steps for the DC restore

[foggy]

Hussain, please review the thread you've been merged into for details on restoring multiple DCs in a completely new environment. If you still have any questions, feel free to ask here. Thanks!

[HussainMahfood]

Thanks for the merge
Executed the following command:

ntdsutil
ntdsutil: activate instance ntds
ntdsutil: authoritative restore
ntdsutil authoritative restore: "restore database" (can not execute this command in windows 2008 r2)


what to do ?

[HussainMahfood]

I am stuck at the authoritative restore. where I need to issue command

restore database. it is not available in windows 2008 r2 any help ????

[HussainMahfood]

created a case number Case # 00256753. will update you once I found a solation.

[foggy]

Hussain, did you have a chance to review this thread from the topic referred above? You do not actually need to perform an authoritative restore at all.

[HussainMahfood]

thanks foggy for reply. the thread is very old referring to version 4 and windows 2003 domain controller. It is not happening with windows 2008r2 and 2012 domain controllers

No updates regard my case yet

[foggy]

Seems that the same procedure still applies both to w2008r2 and w2012:

http://technet.microsoft.com/en-us/libr ... s.10).aspx

Recovering Your Active Directory Forest

Updated: April 25, 2013
Applies To: Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2008, Windows Server 2008 R2, Windows Server 2012

...
2. Because this is the first writeable DC in the domain, you must perform a nonauthoritative restore of AD DS and an authoritative restore of SYSVOL. The restore operation must be completed by using an Active Directory-aware backup and restore application...
...

[HussainMahfood]

Thanks a lot foggy,

Is that mean that I have to restore dc from veeam. Then use windows backup to restore systems state then restore second dc.

Any changes in this process as windows 2012 dc is virual aware system for cloning

Thanks a lot for you help

[veremin]

From my perspective, the procedure described in the above mentioned article should be applicable to Windows Server 2012, as well:

• Perform a nonauthoritative restore of the most recently backed up DC (this process will be handled for you by VB&R), then, an authoritative restore of SYSVOL
• Perform a nonauthoritative restore of second DC.

Thanks.

[HussainMahfood]

Thanks a lot . it worked with me.. just to add information regard authoritative restore (Step) use one of the below methods based on the DC environment.

* DFSR-replicated SYSVOL:

1.In the ADSIEDIT.MSC tool, modify the following DN and two attributes on the domain controller you want to make authoritative (preferrably the PDC Emulator, which is usually the most up to date for SYSVOL contents):
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<the server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=FALSE
msDFSR-options=1
2.Modify the following DN and single attribute on all other domain controllers in that domain:
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<each other server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=FALSE
3.Force Active Directory replication throughout the domain and validate its success on all DCs.
4.Start the DFSR service set as authoritative:
5.You will see Event ID 4114 in the DFSR event log indicating SYSVOL is no longer being replicated.
6.On the same DN from Step 1, set:
msDFSR-Enabled=TRUE
7.Force Active Directory replication throughout the domain and validate its success on all DCs.
8.Run the following command from an elevated command prompt on the same server that you set as authoritative:
DFSRDIAG POLLAD
9.You will see Event ID 4602 in the DFSR event log indicating SYSVOL has been initialized. That domain controller has now done a “D4” of SYSVOL.
10.Start the DFSR service on the other non-authoritative DCs. You will see Event ID 4114 in the DFSR event log indicating SYSVOL is no longer being replicated on each of them.
11.Modify the following DN and single attribute on all other domain controllers in that domain:
CN=SYSVOL Subscription,CN=Domain System Volume,CN=DFSR-LocalSettings,CN=<each other server name>,OU=Domain Controllers,DC=<domain>
msDFSR-Enabled=TRUE
12.Run the following command from an elevated command prompt on all non-authoritative DCs (i.e. all but the formerly authoritative one):
DFSRDIAG POLLAD

Or

FSR-replicated SYSVOL:

1.Click Start, and then click Run.
2.In the Open box, type cmd and then press ENTER.
3.In the Command box, type net stop ntfrs.
4.Click Start, and then click Run.
5.In the Open box, type regedit and then press ENTER.
6.Locate the following subkey in the registry:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\NtFrs\Parameters\Backup/Restore\Process at Startup
7.In the right pane, double-click BurFlags.
8.In the Edit DWORD Value dialog box, type D4 and then click OK.
9.Quit Registry Editor, and then switch to the Command box.
10.In the Command box, type net start ntfrs.
Quit the Command box.

[veremin]

Thanks, Hussain, for coming back and updating the topic with the valuable information; much appreciated.