DC replicas at DR site?

[zoltank]

My office is in two geographic locations separated by about 3 miles, and we've decided to use the other office as our DR site. We are replicating business critical VMs from the HQ site to the DR site. The sites are connected by a 100m EPL circuit. We run a flat network so both sites are on the same network (no subnet).

At HQ we have two DCs and at the DR site we have one DC. We are replicating the two HQ DCs to the DR site.

Assuming we have a disaster and need to bring up the replicas at the DR site, how should I handle the domain controllers? If the source Veeam server is lost in the disaster then I can't gracefully failover and will need to manually turn on the replicas. At that point will the replica DCs still come up in non-authoritative restore mode? Or should I not bring them up, do a metadata cleanup on the running DC at the DR site, and create new DCs? What would be best practice for this scenario?

[veremin]

I'd say that the best option would be to have a VB&R server at DR site to manage replication activity. In case of disaster, you will be able to failover VMs gracefully; DCs will be restored in non-authoritative mode and be synced with the remaining controller.

Thanks.

[zoltank]

I already have a VB&R server at the DR site, but I didn't see anything in the documentation about making the HQ VB&R server talk with the DR VB&R server.

Do you know mean the DR VB&R server would handle the replication jobs entirely on its own?

[zoltank]

I had time to test the replica DCs today, and found out they automatically came up in a non-authoritative restore mode without Veeam's assistance and without failing them over.

While I'm going to do some additional testing in my lab, it looks like I can bring them up normally alongside the existing DC and they should work just fine.

[bahooo]

Hi,
It should work in DR site. Also, do not forget to backup your configuration file after you changed something. This will lead you to restore correct restore file.

[veremin]

I already have a VB&R server at the DR site, but I didn't see anything in the documentation about making the HQ VB&R server talk with the DR VB&R server. Do you know mean the DR VB&R server would handle the replication jobs entirely on its own?

I was more talking about making VB&R in DR site responsible for replication activities solely, not about communication between HQ VB&R and DR VB&R.

Having VB&R in DR site would guarantee that in case of disaster you would be able to execute required commands smoothly (failover, failback, etc.).

I'd suggest to failover DC with the use of VB&R console (not outside of it), so that backup server is aware of the changes done and can initiate failback operation, should need be.

Thanks.

[lando_uk]

Reading lots of these confused DR posts and having many questions myself. I feel Veeam needs to really make an effort going forward to simplify their DR options, integrate failover VBR servers into a single solution so a master VBR server knows about secondary VBR servers for DR. Maybe use SQL mirroring for config DB's rather than in-build export/import options. Its ok for experienced staff to restore, but not easy if those staff also went down in the disaster!

For example, our Commvault DR recovery is really straight forward compared to Veeams, it just uses 2 servers/mirrored SQL and to failover, basically just a DNS change is needed, easy to document and also to test.

[unsichtbarre]

This thread seems to loose site of the purpose for the Veeam Proxy.

I think it is a tremendous mistake to keep Veeam "Servers" at production and DR locations. If the Veeam Server is at the DR site and Veeam Proxies are at the production, then all jobs including DCs can backup and replicate in a coordinated manner.

Regarding lando_uk "master VBR server knows about secondary VBR servers", there should be no need for secondary VBR servers, only Proxies, therefore there is only one control panel to deal with.

Thanks,

[veremin]

I'm along the same lines with John here.

Previously, absence of local backup server might have resulted in undesired behaviour, such as mounting backups via WAN during file-level restore and others. But that is not an issue any longer (thanks be to mount server).

Thanks.

[lando_uk]

We are talking about replication, and the general consensus is to still have a 2nd VBR that deals with your reps-vm's. Mount servers are for item restores, and don't help DR.

[veremin]

The idea coined by John is the fact that basically you don't need two backup servers any longer, as everything can be done via DR one. Thanks.

[zoltank]

1. In my case it would handicap me having the VBR server at the DR site because:
a. If the EPL circuit goes down then I'm essentially cut off from my backups.
b. My tape library is at HQ, which would complicate dumping backups to tape for off-site storage and archiving.
c. I find it easier to maintain two separate VBR servers then supporting one doing backups in two locations.

2. In addition, in a situation where the HQ VBR server isn't able to perform the failover would mean the HQ server room has been lost, which means I'm not going to be failing anything back.

[unsichtbarre]

Actually zoltank, I am sure that you would find the tape library remarkable easy to use when attached (mounted) to a Veeam Proxy. In fact it is as transparent as using the library locally to the Veeam "Server"

The argument about loss of the link between Veeam Server and Veeam proxy is a good one, albeit unlikely on a professional-grade ISP. I run all of my Veeam installations with the Veeam Server at the DR location (except in case of bi-directional replication), and my compromise has been to leave a copy of the Veeam binaries and customers license on the proxy. Veeam installs in about 20 minutes (especially if some components are already in place) and you can import the backups if required! This is not an elegant solution, but it is unlikely to come into play.

[zoltank]

Unfortunately, we back up other things than just Veeam backups, so we still need to run Backup Exec in parallel with VB&R.

[MGT1981]

Regarding the DC's

I have always found it best to just stand up a few Live DC's at the DR site and let AD replicate through built in methods. IMHO you want to do everything you can to avoid having to do a lights out AD restore. Yes the process can be easy and quick but it can also be hard and time consuming based off of things like backup timestamps, USN rollbacks, and journal wrap issues.

If you can put yourself into a position where you have 2 working DC's at your DR site, that should be enough to get your infrastructure started, from there doing a metadata cleanup and promoting 2 new DC's with the same name/IP address as the ones that failed is not a complicated thing.

Remember, if you ever find yourself in the position of having to do a complete failover to your DR site you are going to have so much going on you are not going to want to be spending hours making your AD environment work. Assuming you are starting from a clean baseline, a few failed DC's in AD is not a big deal to clean up and it is certainly more cut and dry than messing around with restores.

Also somewhat off topic but, worth mentioning. you say your DR site is 3 miles from the main site. Are you also keeping backups off site at a third location ? Just something to think about, if the main site goes down due to some natural disaster, or even a medium scale power outage. there is a chance your DR site could go down as well if its only 3 miles away

[matt_778]

Concur mostly with @MGT1981.
Design it so that you can test the failover/failback back any time, not just in case of disaster. When I did the same, (2 DCs in prod (replicated), 1 DC in DR) and I stood up the replicas during a DR test, then rolled back I got USN issues with the domain which required a Microsoft support call to resolve.

I would be more in favour of having light, single purpose DCs at Prod and DR, and do not replicate the Prod DCs, just rely on the DR DC to get you through - you then also might need to consider DHCP/DNS etc, perhaps setup HA DHCP between DR and Prod