Replicate from backup copy with CBT?

Availability for the Always-On Enterprise

Replicate from backup copy with CBT?

Veeam Logoby kevinnaughton » Mon Feb 06, 2017 5:21 pm

I would think this is a common scenario but haven't seen it addressed.

- We backup to disk nightly
- We run a backup copy job to our remote site repository to satisfy off site backup storage
- That site is also our DR site so we replicate to the DR hypervisors there as well

We would like to replicate from the backup copy and eliminate the need to either hit production 2x or transmit data 2x across the WAN, but it seems you cannot trigger replication job from completion of backup copy job. We have tried timing it to schedule the replication job using the remote backup repository as a source for replication, but if that timing is off, files are locked and jobs fail. When those fail, the CBT tracking is messed up and recalculating CBT seems to take longer than re-replicating from scratch even across the WAN.

What is the best way (chaining of jobs) to accomplish a local backup to disk, backup copy to remote site and then replication to remote site without causing source file contention or transmitting data 2x and schedule it to run as soon as backups are complete? The cleanest way we've found so far is replicating from the local backup which means we send the data 2x, once for replica and again for backup copy.

Thanks for any guidance!
kevinnaughton
Enthusiast
 
Posts: 56
Liked: 3 times
Joined: Tue Jun 05, 2012 4:02 pm
Full Name: Kevin Naughton

Re: Replicate from backup copy with CBT?

Veeam Logoby Andreas Neufert » Tue Feb 07, 2017 9:20 am

At the Replica Virtual Machine Definition there is a "Source" button. You can select there the Repository with the BCJ backup files. That should work.
Andreas Neufert
Veeam Software
 
Posts: 2209
Liked: 363 times
Joined: Wed May 04, 2011 8:36 am
Location: Germany
Full Name: @AndyandtheVMs Veeam PM

Re: Replicate from backup copy with CBT?

Veeam Logoby foggy » Tue Feb 07, 2017 11:21 am

I believe Kevin is already doing this, but source backup files are locked - is that right?
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Replicate from backup copy with CBT?

Veeam Logoby kevinnaughton » Tue Feb 07, 2017 3:23 pm

Yes, I tried that. The problem is that you can't trigger a replica job from a backup copy job. So, I have to try and time it and if the job starts too soon the file is locked by the backup copy. If I put too much of a delay in then I am delaying the refresh of my recovery point. Ideally, I'd be able to chain them together as: Backup > Backup copy to remote site > replicate at remote site from backup copy. And have each step triggered immediately upon completion of the prior.
kevinnaughton
Enthusiast
 
Posts: 56
Liked: 3 times
Joined: Tue Jun 05, 2012 4:02 pm
Full Name: Kevin Naughton

Re: Replicate from backup copy with CBT?

Veeam Logoby tsightler » Tue Feb 07, 2017 5:13 pm

Yes, this is a challenge with the current setup today. So far the only way I've been able to make this work reliably is to configure the replica job with a manual schedule and use a Powershell script that monitors the BCJ and then triggers the start of the replica job only after the BCJ is finished.
tsightler
Veeam Software
 
Posts: 4802
Liked: 1760 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Replicate from backup copy with CBT?

Veeam Logoby kevinnaughton » Tue Feb 07, 2017 5:18 pm

Tom,

Thanks for the idea and glad to hear that works reliably. Would you mind sharing your script if you have it handy? I'm going to really delve into PS scripting any day now (I've been saying for a couple years) :?

Thanks!
kevinnaughton
Enthusiast
 
Posts: 56
Liked: 3 times
Joined: Tue Jun 05, 2012 4:02 pm
Full Name: Kevin Naughton

Re: Replicate from backup copy with CBT?

Veeam Logoby foggy » Tue Feb 07, 2017 5:32 pm

It could be a simple command that triggers the replication job, just configure it as a post-job script of the backup copy job. You can find examples on this forum and in the PowerShell reference.
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Replicate from backup copy with CBT?

Veeam Logoby tsightler » Tue Feb 07, 2017 6:17 pm

In my experience it doesn't work as a post-job script in the BCJ because the post-job script for a BCJ doesn't run until the interval expires (unless this behavior has changed). I wish this wasn't the case, and perhaps this has changed in 9.5 as I haven't tested it, but when I set this up I had to resort to something a little more sophisticated.

The script is actually a pre-job script for the BCJ, it then sits running in the background monitoring the progress of the BCJ job and tries to determine the status when all task are done and it is otherwise just waiting for the next interval, it then triggers the replication job. I haven't actually tested this with 9.5 however, but this was the only way I could get this to work reliably with 9.0. I'll test it again in the lab and, if it works, I'll share the script.
tsightler
Veeam Software
 
Posts: 4802
Liked: 1760 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Replicate from backup copy with CBT?

Veeam Logoby foggy » Wed Feb 08, 2017 11:24 am

tsightler wrote:In my experience it doesn't work as a post-job script in the BCJ because the post-job script for a BCJ doesn't run until the interval expires (unless this behavior has changed). I wish this wasn't the case, and perhaps this has changed in 9.5 as I haven't tested it, but when I set this up I had to resort to something a little more sophisticated.

Tom, this looks strange, since the script should run after all data processing tasks complete. It waits for the interval expiration only in case there are any pending tasks. And this was not changed since previous versions.
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Replicate from backup copy with CBT?

Veeam Logoby tsightler » Wed Feb 08, 2017 3:45 pm

The problem is, this is only the case if every single VM in the BCJ works properly so, for example, if you have a VM that for whatever reason fails to be backed up (perhaps a VSS failure or whatever), and it is included int the BCJ, then the BCJ will wait forever for that restore point even though it's never coming. This wasn't an acceptable solution in the cases I was working with, but I agree it might be OK in the simpler cases and if you don't have many failures.

The script I had created tried to be a little smarter, detecting when the job was otherwise idle and starting the replica job anyway, however, it doesn't seem to work with 9.5 so perhaps the simple one-liner is the easy answer for now.
tsightler
Veeam Software
 
Posts: 4802
Liked: 1760 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Replicate from backup copy with CBT?

Veeam Logoby foggy » Wed Feb 08, 2017 3:48 pm

tsightler wrote:...if you have a VM that for whatever reason fails to be backed up (perhaps a VSS failure or whatever), and it is included int the BCJ, then the BCJ will wait forever for that restore point even though it's never coming.

Ah, right, in this case this VM is indeed pending and the script will not be triggered until interval expires.
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Replicate from backup copy with CBT?

Veeam Logoby kevinnaughton » Wed Feb 08, 2017 6:00 pm

I'll see if launching as a post job script after the BCJ works for us. Otherwise, guess I'll stick with using the initial backup job as source and just send it twice over the WAN link. Thanks for all the suggestions.
kevinnaughton
Enthusiast
 
Posts: 56
Liked: 3 times
Joined: Tue Jun 05, 2012 4:02 pm
Full Name: Kevin Naughton

Re: Replicate from backup copy with CBT?

Veeam Logoby foggy » Thu Feb 09, 2017 9:46 am

In the latter case, make sure you're using the same pair of WAN accelerators, this would allow to avoid sending similar blocks twice.
foggy
Veeam Software
 
Posts: 14904
Liked: 1096 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot] and 18 guests