How to chain Backup, Backup copy & Replica from copy?

VMware specific discussions

How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Mon Feb 22, 2016 10:03 am

I'm trying to achieve the above as smoothly as possible but I seem to keep hitting situations where either the replica job interrupts the backup copy job (because it's still running) or the backup copy job interrupts the replica job (from the backup copy) for the same reason.

Am I missing some kind of trick to get all of them playing nicely, e.g. I'd love to start the replica ASAP after the backup copy job completes?
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby Shestakov » Mon Feb 22, 2016 11:22 am

Hello Paul,
How often do you want to replicate? How much time do the jobs take now?
You may use backup windows to separate the jobs or push them start one after another with a post-job script.
Thanks!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Mon Feb 22, 2016 11:32 am

For this particular task, once-per-day is sufficient (based on the one backup/backup copy per day). The main requirement is to minimise WAN traffic.

The thing that confuses me is that the Backup copy job is a "continuous" job & I've had occasions when I wanted to seed a new replica (at the remote site, from the backup copy also at the remote site) but couldn't because the files were "locked". It seemed I had to first disable the copy job, then run the replica, then restart the copy job again (which would then obviously say all restore points already copied, waiting for next).

What makes it more difficult is that we've migrated our DR site to a new site so timings are still being finalized, e.g. I'm now also using Wan-x so waiting for things to settle down a bit.

Can someone please clarify when a backup copy file is "really" locked or is it always the case that to do a manual replica the copy job must be stopped first?
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby Shestakov » Mon Feb 22, 2016 11:51 am

To minimize WAN traffic you can use WAN Acceleration.
Backup copy files are locked during merges. Since its chaining works as forever forward increment, when copying is finished, the job merges .vbk with the oldest .vib
If you perform replication and backup copy once a day, that should not be a problem.
How long do the jobs take in average?
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Mon Feb 22, 2016 11:55 am

Understood, so next time I try a manual replica (from backup copy) I'll make sure there isn't a merge going on at the time (and shall make notes as well).

I don't currently have an average as I've had some (self-inflicted, network-config-related) issues and we only switched over last Wednesday. I think after today things should stabilise & I should start getting a more accurate idea of job runtimes to the new location.

(We've changed from sending backup copies to one site over 100Mb link to now going to a different site over a 20Mb link so old info no longer applies).
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby Shestakov » Mon Feb 22, 2016 12:08 pm

I see, let us know the results you have once everything is settled.
With 20Mb link and jobs overlapping issues I`d definitely use WAN acceleration.
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Tue Feb 23, 2016 11:18 am

Ok, I had a reasonable job run last night. Looks like the copy jobs from one site took about 4:15hrs & the opposite direction took about 3:45. One thing I did note was that the primary-to-secondary site job seemed to have been interrupted after the first 5 (of 22) VMs or so and was 23% into a "merge" process when I checked on it last night.

I initially thought that perhaps the opposite direction job starting at the secondary-to-primary site, which is configured to use the exact same WAN accelerators, but the opposite way around, might have caused the copy job to stop (perhaps due to only a single WAN-x job being able to run at a time) but from what I was reading this morning at veeam-backup-replication-f2/wan-accelerator-concurrent-jobs-t18962.html I could be reading incorrectly but is it the case that WAN-x's going in opposite directions can potentially process a simultanous job (i.e. a single job in each direction at the same time) or not?

In any case as it seems I have around a 4hr window for each site, I'm simply going to stagger the copy jobs so they don't run at the same time (even though neither is saturating the bandwidth). if it becomes an issue I can potentially create further WAN-x's on each side, but they would be VMs rather than physical so I'm not sure if the performance would be sufficient to make it worth while.

However, a *BIG* discovery I made this morning was that when i manually started my replica job, (from primary-to-secondary site) because the backup copy job completed about 06:30 this morning & replica-from-backup-copy was scheduled for 14:00 this afternoon, it replicated a couple of VMs but said that there were no new restore points for the remainder. I knew that there should have been so looked deeper and found that a bunch of replicas were effectively running 2 days behind, as far as restore points were concerned.

Again, I immediately suspected locked backup files so I disabled the copy job (which was sitting "idle") & retried the replica job again and suddenly a bunch of replications started kicking in, i.e. all the ones that were behind in the restore points.

So, I'm still struggling to understand what I'm doing wrong, or what incorrect expectation I have when all I want to achieve is to get a backup copy to the destination site (now working well) and THEN a "latest restore point" replica to be generated from that backup copy restore point.

What am I missing please?
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby Gostev » Tue Feb 23, 2016 9:40 pm

Hi, what is the support case ID for this issue?
Gostev
Veeam Software
 
Posts: 21396
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Tue Feb 23, 2016 10:21 pm

I haven't logged one (yet) as it initially started out as a "how do I..." question but if what I'm describing is unusual then I'll log a call for it.
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby Gostev » Tue Feb 23, 2016 11:38 pm

Frankly speaking, it just becomes hard to follow what's happening when scenario explanation starts reaching "wall of text" size. Forum posts are not ideal for that. It would be so much easier for one of our engineers to just take a look at your setup live, and then confirm through debug logs what is really happening (what locks the files).

But in theory, Backup Copy jobs should be releasing backup files as soon as they are done transferring restore point for the current copy interval.
Gostev
Veeam Software
 
Posts: 21396
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Wed Feb 24, 2016 3:41 pm

Same problem again today so have now logged it as 01708645
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly

Re: How to chain Backup, Backup copy & Replica from copy?

Veeam Logoby pkelly_sts » Thu Feb 25, 2016 11:47 am

Initial view from support is that it is all normal but I disagree. Iooking at things again today I can see that the backup copy job is partially out of sync with the parent backup job, i.e. about 4 VMs are latest restore points but the remainder are a day behind. The replicas seem in sync with the backup copy so I'm not sure why the copy job is going out of whack.

I've just uploaded another set of logs from the last 1 day for support to look into, as well as a bunch of screen grabs illustrating what I'm talking about...
[New Sig: PLEASE get GFS tape support for incrementals!!!]
pkelly_sts
Expert
 
Posts: 504
Liked: 55 times
Joined: Thu Jun 13, 2013 10:08 am
Full Name: Paul Kelly


Return to VMware vSphere



Who is online

Users browsing this forum: Yahoo [Bot] and 22 guests