V6 Replication: Best way to design for source site failure

[jeepdude123]

Scenario:

Source site - ESXi5 hosts, Veeam B&R PHYSICAL server, proxy, repository, local backups for VMs hosted at Source site
DR site - ESXi5 hosts, Veeam B&R PHYSCIAL server, proxy, repository, doing local backups for VMs hosted at DR site
Need to replicate 20 VMs from source site to DR site
Replication from source to DR target works as expected setup per documentation
Documentation always refers to having B&R server and jobs setup at source site, pushing replicas from source proxy over WAN to DR site proxy and DR hosts
This approach obviously does not account for source site failure, meaning source site AND source B&R server is dead - how do you kick off failover of replica VM at DR site when no jobs or knowledge of that replica VM are configured at DR B&R server?

*Checked FAQ and historical threads, doesn't seem to be a consistent V6 best practice of what people do in real world*

Seems like a total hack to somehow replicate the Veeam DB or Veeam B&R server (remember mine is physical) with some other mechanism to DR site.

Is it just a simple matter that to handle a source site failure, configure all the replication jobs at the DR site? Seems simple enough, but why wouldn't the documentation say that and why wouldn't this be THE basic replication design. The reason we have a DR site in the first place is for when the source site goes offline (a disaster!) and all source sites machines are DEAD.

Looking forward to real world design, Veeam B6 best practice etc.

[spgsit5upport]

Keep the copy (backup, daily?) of the SQL database safe at DR site & you are done

Seb

[Vitaliy S.]

...or you can perform failover to any restore point manually via vSphere Client, no need to have a Veeam backup server installed.

[jeepdude123]

...or you can perform failover to any restore point manually via vSphere Client, no need to have a Veeam backup server installed.

Manually huh? So Veeam B&R doesn't provide a solution for source site failure? For real? Seems like a half baked DR product if the only solution in a source site failure is to ditch all of Veeam's marketed features (failover, failback, commit etc) and just use Vsphere client. I'm getting even more conflicting information from a Veeam engineer I'm working with offline. Frustrating.

[jeepdude123]

Keep the copy (backup, daily?) of the SQL database safe at DR site & you are done

Seb

How am I done? So if I manually backup the Veeam db "daily", it's going to just "work" when I fire up Veeam B&R and point it to a DB that doesn't contain all the latest replica job meta data that has been created after the backup? This just seems like a hack. If that is the only solution then it just seems like it's not a Veeam feature to handle source site failure - period.

[Vitaliy S.]

So if I manually backup the Veeam db "daily", it's going to just "work" when I fire up Veeam B&R and point it to a DB that doesn't contain all the latest replica job meta data that has been created after the backup?

SQL database does not contain job's metadata (it stores your backup server/jobs configuration), and yes it is absolutely fine to re-use this database with a new Veeam backup server installation. Another thing that most of our customers do today is replication of Veeam B&R itself. Thanks!

[pcrebe]

I think you must replicate the Source site db to the DR site at the end of all jobs with post activity cmd. In case of Source site full disaster you need restore the db to another B&R server on the DR site and perform failover with GUI.

[tsightler]

First of all, with V6, there is no architectural limitation on where you place the Veeam server, it can be placed at either the source or destination site with no impact on the replication. The important thing for replication is the location of the proxies and repositories. For V6 there should always be a proxy at both the source and target, and there should be a repository located on the source side (the repository can the the same system on which is used as the proxy for the source side).

If you feel better having the Veeam server available after a complete loss of the source site, then you should by all means place the Veeam server in the DR site. If the Veeam server is being used only for replication (not backups) this is actually the "best practice".

However, if the Veeam server is being used for both backups and replication, it might be better to keep the Veeam server at the source location since a network outage would cause local backups to work. For some customers, the best solution is to have two Veeam servers, one in the source site to handle backups, and a second in the target site to handle replication. This has the advantage that there is a server in the DR site to perform failover, and also to use for backups once a failover has taken place, an issue often overlooked in the DR design.

Manual failover is generally suggested only as a last resort because this does mean that you would loose some of the features such as Re-IP, however, the process overall is very easy since the VMs themselves are native VMware VMs simply waiting to be powered on. Even in the event of a complete manual failover you can still failback relatively easily by using replica mapping.

Having a good replication design is important and it's good to see that you are thinking about all of the options and possibilities. Veeam provides a lot of flexibility in it's approach here. For a physical solution you can place the Veeam server wherever you like, source or target, you can virtualize the Veeam server itself and replicate it, using the physical server as a proxy, you can use SQL native replication, you can use SQL native backups to backup the database/transaction logs to the remote site. It's all based on your goals and desires.

[jeepdude123]

Sounds good, so since I have 2 physical veeam servers already, the one at DR site should just handle the replication jobs which would solve the the source site failure scenario. As you said, the DR server will also do backups in an actual site failure.

Now seems like the question is "what is best way to backup Veeam server". Sounds like the slickest way is to have a Virtual Veeam server at each site (physical server could be proxy + repository), the virtual Veeam server replicates itself for backup to a stand alone host with DAS (separate from our main vmware cluster / SAN) so if the production SAN blew up, we have a virtual Veeam replica sitting on a separate box ready to go.

My original thought of going physical Veeam server was following a best practice to have your backup server on separate hardware from your production VMs etc, but you'd have to backup that physical Veeam server (at least SQL db) to deal with a Windows server corruption or hardware array data loss issue, seems more complicated than having Veeam as virtual.

So changing gears to running Veeam as virtual, any hidden real world gotchas? I know that features seem to be the same, but is one far more straight forward than the other? Physical always seemed like it would be easier to troubleshoot perf issues etc since it's outside of VM land and much less resource contention etc.

Thanks for responses, Veeam seems pretty flexible and many ways to do things, but in my experience there is a design or solution that is usually superior based on either reduced complexity or ease of troubleshooting etc. I'm trying to finalize that design and meet all data protection and DR likely scenarios. One thing I can say is my experience with Vmware SRM and array based replication was awful and I'm pretty please with the Veeam simplicity.