What happens if a DR event occurs mid-replication?

[nickf]

We've successfully done a couple of DR tests using Veeam replicas but each time we do a test we disable the Veeam replication jobs in advance so that when we cut the WAN link between sites we don't risk corruption on the replica. However it occurs to me what would happen during a real DR if the WAN link or source site died mid-replication job, Would the replica be usable and is there any special procedure to use when bringing it up? We do keep 3 restore points per replica but I'm not sure if that would help if the main VMDK is corrupt do to only partially updating?

[Vitaliy S.]

Hello Nick,

Yes, that would be usable, just not the latest restore point (which would be incomplete obviously). You may want to follow one of the scenarios below to perform a failover:

1. Install Veeam backup server on the DR site and use it to perform both replication jobs and a failover to a previous point in time. The restore points will be still usable even if your WAN link crashes down during the replication job.

2. You may backup your Veeam SQL configration db located on the Main site using native tools such as SQL Management Studio, and then re-use this db on the DR site with a new Veeam backup server installation.

Hope it helps!

[tsightler]

This is pretty much why I'm a fan of "pull" replication when using Veeam. With push replication there is a risk that the replica would not be usable and you'll be left in an unrecoverable state. With "pull" replication (the Veeam server is on the DR side) you can easily roll back to a previous restore point.

If you want to use "push" replication and still be able to revert to a previous restore point then there are really two option:

1. You can install Veeam on the DR site, leave it in a "stopped" state, and use SQL replication to keep the Veeam DB at both sites in sync. In the event of a failure you can simply start Veeam at the DR site and use this instance to revert to a previous restore point. I'm not such a big fan of this because of the complication and dependencies it creates (you need working Veeam and vCenter at the DR site before you can power on VM's), but it works.

2. We "cheat" by taking storage array snapshots on our DR side prior to the start of replication. That way, if a replication cycle is interrupted by an outage we just revert the storage array snapshot and we can power on the VM's. We find this simpler, and it has no reliance on any of the rest of the infrastructure (like vCenter or Veeam).

[lobo519]

Sorry to bring this back from the dead but it beats trying to explain the situation again.

Is there a performance hit if I choose the "Pull" replication VS. "Push"?

[Vitaliy S.]

There shouldn't be any noticeable performance impact.

You may have a look at some Pros and Cons related to installing VBR on-site or off-site to have a clear picture:
http://www.veeam.com/forums/viewtopic.p ... 8738#p8738

[lobo519]

great - additional question
Looking into the Replicate sql server db to remote site for replication failures idea...

If I try to hook up the DR site Veeam install to the replicated Veeam database I get the follow error

Configuration database is in use by the Veeam Backup console Installed on "Local installation" To reconnect database to this console please run Veeam backup setup program.

I reinstalled but I am getting the same error. Am I missing something?

I can submit a support ticket if needed.

Thanks!

[Vitaliy S.]

No need to submit a support ticket, here you go the resolution for your problem: Replicating Veeam Database

[lobo519]

No need to submit a support ticket, here you go the resolution for your problem: Replicating Veeam Database

just found this - works like a charm!

Thanks!!!

[RumataRus]

[merged]

if a replication job runs and due to a malfunction it is unexpectedly interrupted, will the replica roll back to previous "good" state?
In other words, is it possible that the replica will be corrupted in case of the replication job unexpectedly fails?

[RumataRus]

Sorry, but I have a different situation.
I have no DR site. I do local replication and I have Veeam Backup installed in a VM.
All my hosts are connected to one stack of switches.
I modeled the situation of failure of one of the switches (I just turned it off. )
In any case, there is a timeout until the stack is restored.
If at this time replication job was running, damage of the replica occurs.
Also I have experienced that if host isolation occurred in HA Cluster (host with Veeam Backup) while replication job was running, the replica has been damaged too.
Are there ways to avoid this?

[Vitaliy S.]

Well...this topic does cover your first question. Just read back through my first post and the first sentence, in particular.

Veeam will try to repair this replica automatically on the next incremental job run, though please keep in mind that you can also failover to a previous rollback with Veeam failover wizard even before the repair happens.

[RumataRus]

Mm...
I have the feeling that before I can answer you, I need to do some experiments.

[dinger76]

[merged]

Hi Guys,

I've not gone and tried this but just wondered what you believe would happen and the best way to avoid the situation.

If you are performing replication of your virtual machines between two sites, once a day overnight. In our scenario, we keep two rollbacks at our destination location to keep the storage requirement low. Suddenly there is an issue at your source site, during the replication process and therefore the replication task fails.
Would this have deleted the full replication that would be required to perform a failover to the destination site, or can you still failover to one of the other rollbacks?

Is there a way to make sure that we always have a safe full vm at our destination site to avoid issues like this occurring?

Thanks

Chris

[J1mbo]

See above...

please keep in mind that you can also failover to a previous rollback with Veeam failover wizard even before the repair happens

[Gostev]

v6 is going to be even nicer: no restore point will be left inconsistent in case of failure in the midst of replication.

[darren.wardle50]

Gostev

Can you give us a little more detail on how replication has changed in v6 in regards to the handling of mid replication failure and the reliance of the SQL database for recovering our servers in this scenario?

Our replication server is at the source site with full ESX at the destination due to the need to run an initial replication to USB drive as our 10Mb link can't handle the initial replication of the 1TB+ of VMs we are replicating.

We do currently replicate the Veeam database using SQL replication but we have concerns that the recovery process using the database copy adds extra steps and has a much higher likelihood of failure resulting in servers that will not boot correctly.

[Gostev]

Hi Darren, we will provide extensive details on the upcoming webinar (see sticky topic on this forum). In fact, the very next webinar is dedicated to replication enhancements. Be sure to register and attend. If you cannot attend, you will get a link to recording within 48 hours after the webinar. Thanks!