Backup Copy Jobs Question

[ddubois]

If I have a ton of VMs that backup at least daily and have a copy job setup to copy those backups to another repository. I am trying to find a way to grab some of those backups nightly for high priority VMs and weekly for low priority VMs. If the cycle is set to daily for the important VMs then it grabs the daily of all the others as well that I only want weekly copies of. Is there a way to control this?

[DGrinev]

Hi Derek,

If I get you right, you can create two backup copy jobs where the first one will contain only important VMs for nightly cycle and the second one for the rest with the interval of 7 days.
Please keep in mind the backup copy job isn't copying backup files, but moves VM data blocks from the backups. Thanks!

[ddubois]

Right, it not touching the live VMs but is getting changed data and synthetically staging it to the other repository, correct?

[nitramd]

@ddubois, a backup copy job does not "touch" the live VMs; it works with the VM files found on the repository, which, of course, was generated by a backup job.

Hope this helps.

[ddubois]

Great. The problem this brings me to that I have posted on here and got merged into another feature request post on job priority, is that if I split it into two jobs, then there is no way to tell Veeam to run the daily jobs first and either the weekly job (that has a huge amount of VMs in it) either take up all the available concurrent tasks, or if I up the tasks, the weekly one then seems to take 95% of the bandwidth causing the small number of VMs in the job to fail... Was hoping that maybe if I worded my problem different that someone might have a good workaround instead of it getting lumped into a 5 year old feature request...

[nitramd]

@ddubois , I'm not sure I understand what you're saying. However, I'll give it a go:

- Split your large backup copy job (yes?) into two backup copy jobs; one for critical/high priority VMs (runs daily) and the other backup copy job will be run weekly.
- Schedule your daily backup copy job to copy every one day with whatever start time you want, e.g., 11:00 PM.
- Schedule your weekly backup job to copy every 7 days with whatever start time you'd like, e.g., 3:30 AM.

If you decide to go this route, I would highly recommend that you have a firm grasp on how much data is being copied on every backup copy job run and how long it takes to complete. This information will help you determine when run the weekly backup copy job. The hope here is that you avoid resource contention on your repository as much as possible.

You may find that you may have to experiment, if possible, to find out what works best for you.

Good luck.

[ddubois]

Yes, I now have it broken into two jobs like you are referring to. However, there is no way in Veeam to prioritize a job over another so, it always seems like the "weekly" job that is currently running, grabs almost all of the available bandwidth and is preventing the "daily" job from completing until later in the week when the weekly job finishes. From my previous posts about it, I got merged into this post but was hopefully seeking a workaround... veeam-backup-replication-f2/enhancement ... 18019.html

[nitramd]

Ok, I have a little bit better idea, thanks.

It sounds like your running into hardware limitations, i.e. network, CPU, memory (possible).

My first question is how much total data are you backing up with the weekly job?

Next, what type of repository do you have (make, model)?

[ddubois]

local Dell server with two 64TB repositories, remote dell server connected by a 10Mbps fiber link with a 64TB repository. Both servers have 24 cores and 64GB RAM. I have a blackout period set from 6AM to 6PM Monday through Friday. It is a large amount of data for the weekly job and pegs the max on the fiber at all times. I would say that the weekly job is getting about 1MBs of the bandwidth leaving about .1-.2 for the daily job. There is no way to say, "Hey, put the weekly job on hold until this daily job finishes". Which would take about 4 hours. The weekly job completes in about 4 days and we are still adding servers to it. The daily job just includes two SQL servers that are backends for our applications and in a disaster, do not want to loose more than a day. We are backing up about 14TB of data total. For the backup copy jobs, we are adding servers one by one until we hit our limit. The only gotcha is that the weekly job smothers the one we need to run daily.

[nitramd]

10Mbps fiber link?

Is it possible to route your critical daily job to a different repository?

[ddubois]

Unfortunately, our remote server across that line is our only remote repository. I hate the idea of having to setup another server to act as a repository in another one of our sites. In typing this, I just realized a workaround that had escaped me until now. I can block out more time on the weekly job to let the daily one have a few hours to run before the other starts back up. Can't believe that I didn't think of that sooner. I just wish there was a way to tell Veeam, let this job take precedence when it starts.

[nitramd]

Great! That's advantage of "talking" about a problem. You eventually realize something that you had not before.

Good luck.

[ddubois]

Thanks for your help... I do wish there was a way to let them both run and throttle them or prioritize them... Maybe in release 10...

[mountainSE]

If you have the Enterprise Plus version, this is a perfect situation for use of the WAN Accelerator feature.

[ddubois]

I was told that the direct would be best for 10Mbps or faster connection by support... But I might just give it a shot and see how it does!

[mountainSE]

If you are getting the full 10Mbps pipe for the Veeam jobs, then yes, direct would be best. You just mentioned that you were only getting 1MBs throughput, so I wasn't sure.

[jmmarton]

In the past the best practices guide suggested WAN Acceleration may be beneficial up to 100 Mbps of bandwidth. Today the guideline is more like 50 Mbps, but also notes "your mileage may vary."

https://bp.veeam.expert/resource_planni ... nment.html

With only 10 Mbps I would recommend at least giving WAN Acceleration a shot. If anything, use Veeam ONE to gauge the savings you're getting.

Joe