Veeam B&R recovery of a domain controller

[Louharle]

Looking for documentation regarding the process of recovering an Active DIrectory domain controller through the use of Veeam Backup and Recovery version 5 or later, utilizing VSS and not inducing a VSS. Areas that I would like clarification on are the process that Veeam goes through to boot in to AD RECOVERY MODE and how does it decide what mode to go in to (Authoritative/Non-Authoritative) and how does it know it's an Active Directory Domain Controller (or is there checkboxes or the like that are available during the restore process). Any guidance / information would be most helpful.

Thanks,

LouHarle

[Gostev]

Hi Louis,

Unfortunately, I will not be able to provide much details on how we do this on public forum, because this technology is our know-how. Please appreciate that we are still the only vendor to provide this feature for image-level backups.

I can address some of your questions above though.

1. DC will always boot in non-authoritative mode, which is exactly what you want most of the time. You should not need to perform authoritative restore in most cases unless your Active Directory corrupts or something because, authoritative restore reverts AD to the earlier point in time and can cause too many additional issues by itself. It is really the last resort. But if you need to perform it from Veeam backup, there are existing discussion with verified procedure posted on this forum that you can refer to.

2. Veeam Backup knows that computer is a DC by querying certain Windows OS configuration parameters from within OS (our application-aware processing logic does that when backup is performed). So, detection is fully automated (there are no checkboxes).

Hope this helps.

Thanks.

[Louharle]

It does, very much. I agree that 99% of the time non-authorative will be the mode you want. I appreciate the fast response!

[habibalby]

Hi,
I'm testing Veeam Backup to backup our Domain Controller. I just did a backup test on test DC and it backed up fine. I deleted the VM, and I restored it from the Backup. When it boots, it booted fine without any issue. I looked at the Event Viewer and I didn't see any errors or warning. Does this mean Veeam restoring DC as an authoritative restore mode? Or I must perform the authoritative restore mode manually?

Thanks,

[habibalby]

Hi,
I have just run my DC 2008 from a Backup and start the machine and entered F8 to go to the DSRM,, it logs fine and asked for the local username. I poovided the local user name of the DSRM but it didn't restore successfully. How can I restore the DC / Run it from Backup?

Thanks,

[Gostev]

Hi, reading through my response above, DCs are restored in non-authoritative mode. The process is automatic, and you should not do anything (just wait until the restored DC automatically reboots). If you are looking to perform authoritative restore (which you really do not want to do in most cases), please search for the existing topic describing the procedure of such restore. Thanks.

[habibalby]

Hi,
I found that when the veeam backup is finished, in the Boot Sequence the Auto Checked on the Active Directory Repair under MSConfig is checked even after second reboot. What I did, I rebooted the machine first and when the DS Restore completes I unchecked this option and the Active Directory starts normally and I was able to login to the Domain.

Veeam should comes with a proper documents on how to Restore DC/ Active Directory.

Thanks,

[Gostev]

There is no documentation needed, as the non-authoritative DC restore process is fully (100%) automated. I am not sure what are you doing and what settings are you looking at and changing, but again, you do not have to be performing any manual steps at all. After being powered on for the first time, the restored VM will boot in safe mode once (the very first boot), perform VSS restore, and then reboot immediately. Until this point, everything is fully automated. After that reboot, the VM will boot up normally, and the restore process is done. All the following reboots (those initiated by user) will be normal as well.

We had automatic DC recovery functionality in the product for 3 years now, it is very polished by now, if it had bugs with it, someone of 25000 customers would have certainly run into them by now. Of course, there might be some environment-specific issue in your case, so if you think that something is not working as it should, feel free to open a support case, and our support stuff will troubleshoot this for you.

Thanks!

[habibalby]

Hi,
As I stated above;
1. First boot it goes into safe mode
2. Second boot it goes into Directory Restore Mode
3. Third boot it goes again into Directory Restore Mode... Humm, here's the trick. It goes into DRM because it doesn't uncheck the Safe Mode and Active Directory Repair in the Boot Option under the MSConfig System Configuration.
4. I initiated the MSConfig and unchecked the Safe Mode boot
5. DC Starts Normally.
6. Set the IP Address again and make sure DNS and name resolution okay.. and reboot to make sure DC is working fine.
7. DC Up and Running.
8. Thanks to Veeam for this Solution

[Gostev]

Got it, at step 2 above something does not work as expected in this case, because normally the process should go from step 1 right to step 5. Do you wait long enough after 1st boot? Does reboot 2 happen automatically? We would want to investigate this if possible (if you have time and desire to work with our support). Thanks!

[habibalby]

Hi,
It doesn't reboot automatically.. I have finished three cigarettes and came back but it didn't that's why decided to reboot it automatically.

Thanks,

[Gostev]

The in-guest process that automatically reboots DC is also the one disabling safe mode boot. Thus, clearly there is something wrong with it in your case, if neither happens.

[habibalby]

Aha, In my opinion to create a KB about these issues and how to overcome such as things when it happened. Luckily I have an experience in backup and restoring DCs.. It came to my mind to see the Boot Sequence and I found the issue. Otherwise I would force to seize the FSMO roles to get it restored.

Thanks any way for your help

[Gostev]

Well, again, we have no reason to believe that the issue even exist in the product. Because it was never reported before in all these years, it is very unlikely. This need to be investigated first. Might be incorrect product usage (application-aware processing not enabled), might be issues with specific DC being broken or having broked security settings, and so on.

[danieln]

Hi everyone,

I have the same issue, one of the 2 DCs is constantly rebooting in safe mode and because I cannot logon on it there is no way for me to disable the boot.ini modifications Veeam B&R does.
Veeam B&R modifies ALL boot.ini boot options, not only the default one, beats me why all.
Hussain, how do I go about letting it boot normally?!
One way would be to attach the disk to a running guest and modify the file, I guess…

For Veeam team: clearly there is an issue some time, just saying 25,000 did not complain ...
I wonder what can it be: maybe the antivirus software may prevent the file edit process, even is safe mode? We use Kaspersky.
In case of a DR event (or a simple VM restore) nobody needs guess work and custom boot scenarios, it should just work.

Thank you,
Daniel.

[Gostev]

Well, that's exactly the reason why we start DC in safe mode before performing VSS restore - Microsoft requires this to make sure no 3rd party tools are running because they can potentially lock Active Directory database and other system files, thus preventing correct restore. Unless of course Kaspersky uses some very deep&dirty integration with OS via some hacks, making it always start (even in Safe mode) - which is a possibility. Our support should be able to determine what exactly prevents the restore logic from working properly, so I suggest that you open a support case at your earliest convenience.

[joergr]

to add this: this even worked in the times of veeam backup 2.0 like charm, veeam is a pioneer in this technology, remembered it and found it with google, take a look, featuring also good video demos ->

http://www.google.de/url?sa=t&rct=j&q=v ... Ig&cad=rja

best regards
Joerg

[Gostev]

Wow, these are like over 3 years old videos now, I remember myself making them shortly after I joined Veeam... what a find, Joerg. I am having a nostalgia moment here I am guessing this post was migrated, because at the time we had different blog, VeeamMeUp.com

[vota]

[merged]

I've a problem with Surebackup of a 2008R2 DC VM.
When i run the Surebackup-Job of this VM it starts in Directory Repair Mode.
This Problem exists since i use Veeam.

But its very interessting:
when i start an Instant-Recovery of this VM and boot it in the Virtual-Lab(booting with change the VMs-Network in vSPhere Client) the VM starts in Directory Repair Mode too - but after i logon and let it some time - the VM restarts normally

Veeam-Version is: 5.0.2.230

Does anyone have some suggestions for me

[vota]

hmm - i think my problem is solved.
i increased the timeout in my applicationgroup and now its working fine - maybe the vm took to long to come up properly?!

i will run a second testrun and give feedback later.

thanks,
BR
Stefan

[oarmandt]

First I must say I use version 6 of Veeam B&R.
I'm fairly new to Veeam B&R. I have one DC in the domain and as far as I understand Gostev earlier in this forum it is not a problem to replicate and restore.
Automated altogether - it should boot in safe mode etc.
When I first power on I'm asked how I will start Windows with Start Windows Normally being selected. This is as if Windows had been powered off unexpectadly - if I now accept Start Windows Normally it will boot up and after login I receive the dialog where to tell why it went down.
This is not as expected according to what Gostev said earlier - so what should I do to geet this right??
As a matter of fact all my servers is starting up in the same way - which seems wrong to me.

[Vitaliy S.]

Hi Owe, actually dirty power off message is expected, you may find more info on that if you follow this link: restore image question

[velowulf]

I will add some weight to the request for documentation on this topic. It took me nearly 6 hours of searching the forum and running test restores in my testbed followed by diagnostics, etc and a support call to Veeam to be confident that my DC restore was going to be effective and not bring down the entire domain.

Gostev: I don't think that anybody is asking for an insight into exactly what you do and how you do it (your intellectual property) but it would be EXTREMELY useful to see a step by step guide into DC restoration. This would answer a number of questions that I have found repeated time and time again on this forum. For example, is the dirty power off screen expected?, should you login to the machine?, how many times will the machine reboot?, do I need to take any additional steps to restore a domain controller?, what do I do if the restore fails or appears to fail? I am across the correct answers for all of these now (if anybody needs help please PM me) but these answers were gained through a lot of hard graft and, ultimately, a lot of crossing of fingers and hoping. This isn't really acceptable when it comes to backing up and restoring machines on your network. A step by step guide to restoration would alleviate all of these concerns.

[oarmandt]

OK - if dirty power off message is expected which startup choice do I take?

Gostev said once -DCs are restored in non-authoritative mode. The process is automatic, and you should not do anything (just wait until the restored DC automatically reboots)

Gostev also said once - After being powered on for the first time, the restored VM will boot in safe mode once (the very first boot), perform VSS restore, and then reboot immediately. Until this point, everything is fully automated. After that reboot, the VM will boot up normally, and the restore process is done

What is to expected and what shall I do??

Thanks

[Gostev]

Paul, I agree it would be nice to have step-by-step for everything at all. But since DC recovery requires no user interaction at all (fully automated), the need for such documentation has not been pressing. Very, very few people dig as deep as you do - 99% just deploy and start using, trusting our expertise that the product will do the job correctly when the time comes (which it does, anyway - as you can see from your testing).

Owe, you don't need to do anything at all. The whole process is fully automated, you should not even open the restored VM console.

Thanks.

[vertices]

I can confirm that I just had the same problem with a DC recovery not finishing correctly. This was with Veeam 6 and ESXi 4.1. Environment is completely standard, single subnet, DCs are single purpose. This was a simple 2008R2 DC/DHCP/DNS server with no other software other than AV. I restored the DC (test scenario as I had never recovered a DC with Veeam before) and it correctly booted into Directory Repair mode. It sat at the login screen for around 10 minutes then auto rebooted. So the restore of AD was successful but every time it booted after that it booted into Directory Repair mode. So I logged in and changed the boot options in Administrative Tools > System Configuration > Boot Tab and unchecked "Safe Boot with AD Repair" and then it booted fine.

So something is definitely amiss here with AD restores. They work it appears, but definitely not fully automated.

[Gostev]

The process is fully automated. There are rare cases when this does NOT work (usually because of 3rd party software), but you cannot really make conclusion that it "definitely never works in fully automated manner".

For example, original poster has the issue with reboot not happening, and he was totally convinced it never happens automatically. However, as you can see, it does happen just fine as expected.

In your case, I would guess the issue is likely to be antivirus preventing our in-guest process from modifying the boot options automatically (and 10 minutes is too much of a wait anyway, this sounds like some internal timeout for the guest process). The process log from guest would confirm this, I suggest that you open a support case for this. If this is found to be the reason, the easiest way to fix this would be to simply add our guest process into the antivirus exceptions.

Thanks.

[vertices]

Can you let us know here what should be excluded from AV so we can go ahead and do that as a precaution?

Also, can you add how long we should normally have to wait once it boots into Directory Repair mode? And how long we should let it sit here before rebooting if it doesn't auto reboot?

[Gostev]

Rob, you should have the issue researched by our support first before we can make such recommendations. This was just an idea one from the top of my head on what could potentially be preventing boot settings update.

As far as I remember, in my lab reboot used to happen in about 1 minute after safe mode GUI appears. This was Windows 2003 DC and a small AD domain with only 2 DCs, so may be Windows 2008 DC would take a bit longer.

[rawtaz]

[merged]

Hi,

I have done a full restore of a few VMs, of which one is a Windows 2008 R2 Domain Controller. It was backed up with VSS support by B&R version 6, and restored to an ESXi5 host.

Now when I boot it for the first time, I am getting in the console the standard black screen saying Windows wasn't shut down correctly, and it offers me to boot in one of the three safe modes, or normally. A screenshot is at http://grab.by/c7Po .

Is this expected, or should the booting happen som other way? I have gotten the impression that Veeam somehow should make the VM boot into DSRM itself the first time, and one should wait patiently for it to reboot after up to 15 minutes, after which it would boot normally. Is this what should happen, or should one boot DSRM manually?

Please clarify what one should expect to happen on first boot, and also any steps one should take in order to get a restored 2008 R2 DC running again correctly. This is for everyones' reference, it wasn't easy to find a clear summary on this topic.

Thanks!