Veeam B&R recovery of a domain controller

[Unison]

Hi James,
So painful! Hopeful you will be able to avoid all of that in future.
can i ask - why dont you enable application aware processing on the DC backup jobs?

It wouldn't take much time for you to create a new veeam job for your DCs only - turn on application aware processing for them - capture the backup......then test your restore process again on your isolated/island hosts.

My DC jobs have application aware processing on them - to ensure crash consistent backups particularly for the AD databases. I quite regularly do a DR test similar to what you are doing......simulate a total loss of the physical environment and restoring everything using just the backups (and our physical veeam server - everything else runs on vmware and Dell R710 hosts).

Be interested to see if you do a quick test with your DCs with the app aware processing - if your DR tests like this become MUCH easier.

[JamesNT]

Hello Unison,

Thank you for replying. We will be trying that type of backup next. However, my question does remain: Does our having to restore the VM's by restoring just their files and importing them make a difference? Your response implies no, but I thought I would seek clarification.

JamesNT

[Unison]

no worries
my answer would be 'no' - wouldn't make a difference or be the cause of what you have seen here - but hopefully a Veeam 'Green Guy' can chip in here to confirm.

VMs are really just a collection of files (as i am certain you know) - which is what makes them so flexible/agile......you should be able to take the files (which make up the VM) and take them to ANY hardware/hyper-visor and run that/those VMs there.....with the same config/resources, those VMs should be none the wiser that they are actually 'somewhere else' and it should not impact recovery at all.

keen to know if your new backups with App Aware improves your recovery/DR tests greatly

[foggy]

Upon restore, it is important to note that the cluster the domain controllers were running on no longer exists. VEEAM wants to, upon choosing to do a full VM restore, look for that cluster and restore to that. Not an option.

James, were you trying to perform restore to original location without actual access to it?

After reading this thread, we noted that we did NOT have Application Aware Processing turned on for the domain controllers. However, given the approach we have to take to restore the DC's, with having to do individual files and then import them using Hyper-V manager, would that make a difference for us? Or are we destined to have to always bring in one DC and remove the other every time we do a restore?

Provided you have AAIP enabled in the backup job, the recovery process in the "all DC's are lost" scenario would be as described in this KB article, regardless the type of restore you perform (full VM restore, VM files restore, Instant Recovery).

[ksauer]

Upon restore, it is important to note that the cluster the domain controllers were running on no longer exists. VEEAM wants to, upon choosing to do a full VM restore, look for that cluster and restore to that. Not an option. So in order to restore the backup, we have to choose to restore the individual VHDX files and config files to a folder on one of the hosts, use Hyper-V Manager to import the DC's, setup a new cluster, then move the domain controllers into the new cluster.

I'm 99% sure this is a known issue and is slated to be fixed in V9U2. The work around for this issue is to simply perform a VM Files restore to manually restore the files rather than leveraging the Entire VM restore option

[JamesNT]

Gentlemen,

I cannot begin to thank you all enough for this valuable information. It appears we will be re-writing a good portion of our disaster recovery procedures and life is going to become a bit easier.

One other question, if I may: In the event of a the scenario where all DC's are lost (and we have only two) or if we just want to set up a new test environment using backups from long ago, how far back can we go with our backups to restore both DC's? Since we are restoring both, can we go back to 365 days ago? Yes, we are keeping backups that long.

JamesNT

[foggy]

You can select any of the restore points and restore to the required date.

[JamesNT]

I'm 99% sure this is a known issue and is slated to be fixed in V9U2. The work around for this issue is to simply perform a VM Files restore to manually restore the files rather than leveraging the Entire VM restore option

Can we confirm this? I can't seem to find the feature that a Hyper-V guest can be restored by choosing Full VM Restore to a different host or cluster anywhere for version 9.

JamesNT

[Regnor]

Team, let's create a support KB on authoritative AD restores with Veeam (step by step). Please coordinate with QC and Support. Thanks!

+1

A colleague worked on a case where a customer had problems with a recovered DC. I didn't find any information on how the restore is done by Veeam and now months later I've found this thread

[Gostev]

Support KB article is already available > KB2119

[foggy]

Can we confirm this? I can't seem to find the feature that a Hyper-V guest can be restored by choosing Full VM Restore to a different host or cluster anywhere for version 9.

Currently the issue is confirmed on vSphere only. Are you selecting restore to an alternate location? Do you have a case open for this?

[Regnor]

Support KB article is already available > KB2119

Thanks

[JamesNT]

foggy,

We choose the option in Veeam to restore only files then use Hyper-V Manager to import the VM.

JamesNT

[JamesNT]

Support KB article is already available > KB2119

The article does not mention when the registry entries may be deleted. Can anyone elaborate?

JamesNT

[foggy]

We choose the option in Veeam to restore only files then use Hyper-V Manager to import the VM.

Have you tried to select restore to an alternate location on the corresponding step of the wizard?

The article does not mention when the registry entries may be deleted. Can anyone elaborate?

You can remove the keys after domain recovery is completed.

[JamesNT]

And when do we know when domain recovery is completed?

Since we have only two domain controllers, is that when both have been rebooted twice, one after the other? Is there a specific entry in the event log we should see?

JamesNT

[foggy]

Yes, after the second reboot and making sure both DC's operate fine and replication works normally.

[JamesNT]

foggy,

Thank you very much.

James

[pidthepiper]

OK everybody dont panic!

Basically management asked me a question, and I was wondering if anyone had actually done it.

Lets say we increased our domain level from 2003 to 2008 and for some reason we wanted to roll back.....

Could we turn off the current DCs and restore our application aware backups of all the DCs from when the functional level was at 2003 and be back up and running?

Random I know........

[foggy]

Yes, please see the KB mentioned above for details on the restore procedure.

[guidoakd]

I'am lost with all the opinions haha!

¿Is it possibly to restore a DC with Veeam? In spite of being replicating with anotherone... when you restore the machine... ¿they re-replicates properly? ¿Anything changes if the DC corrupted is the one what has all the roles?

Today i'm doing the backups with the Security Copy of windows.. with the system state.. ¿Does Veeam replace it?

Thanks

[foggy]

Guido, I've already replied in another thread, please review the KB describing all the DC recovery scenarios and feel free to ask any additional questions, if required. Thanks.

[guidoakd]

Hi everybody.
I am following this KB https://www.veeam.com/kb2119 and in particulary the scenario #3, which is the one you try to restore a damaged/corrupted DC.
After reeboting and making the non-authoritative restore, when i go to the command prompt on DSRM mode i have some issues:
At first, the instance is not activated, so i activate with: "Activate instance ntds"
But the problem appears when I try to restore the database. The line that says:
6. At the "ntdsutil authoritative restore:" prompt, type "restore database" and press Enter.
It returns "invalid syntax". If i go to the help menu, i can see that the option restore database doesn't exists (only restore object and restore subtree). Over there i don't know what to do or write.

Does anyone to how to make it work?

Thanks very much.

[Vitaliy S.]

Hi Guido,

What DC version are you trying to restore? Do you have the same behavior for all restore points?

Thanks!

[guidoakd]

Hi.. my version of DC is 6.3.9600.16384
It is a Windows Server 2012 R2
I've "corrupted" the laboratory DC by using the command SYSPREP.. i don't know if it was a letal weapon haha, but i think it is independent of this command that i can't execute.
(Any other suggestion to corrupt a DC?))

I only tried with one restore point, now im going to try with anotherone.

Thanks

[Vitaliy S.]

I believe "the corruption" method should not matter here Yes, as a first troubleshooting step I would suggest to verify you're using AAIP for the backup job, and then try another restore point. If the issue still persists, then I would recommend contacting our support team for reviewing the debug info.

[millahjovich]

I have just run my DC 2008 from a Backup and start the machine and entered F8 to go to the DSRM,, it logs fine and asked for the local username. I poovided the local user name of the DSRM but it didn't restore successfully. How can I restore the DC / Run it from Backup?

[Vitaliy S.]

if you've tried to follow the guidelines from this KB article and still cannot restore the DC VM, then please contact our support team for logs review.

[ftbug]

Hi, When we restore a DC into a different environment, we are unable to start any DC services and are unable to open sites and services, with the error stating that the domain does not exist or cannot be contacted.

When this DC is replicated into another environment within vsphere, the services start and we are able to clear the AD environment up and resolve the issues, so it seems that the restore is what is causing the issue within active directory

[foggy]

How do you perform DC recovery? Do you have application-aware image processing enabled in the backup job? Please review the KB mentioned above for a better understanding of DC recovery processes.