VM backup order within and across jobs

[aeccles]

I have 3 jobs setup, each with 3 or 4 VMs in them. Sometimes the VM backups queue because they are waiting for resource availability. When this happens, the order in which VMs are backed up is lost. For Example:

Job 1 - 7pm
Server 1
Server 2
Server 3

Job 2 - 8pm
Server A
Server B
Server C

Assume my repository is limited to 1 job at a time.
At 7pm, Job1 kicks off and let say Server2 is still backing up at 8pm... Job 2 now starts and is immediately queued, waiting for resources. When Server 2 finishes, I want Server 3 to start backing up, however Server A backs up next. Other than changing the job times, is there any way to prevent this from happening? And if so, can that change be made without having to reseed anything thing (the issue is with a backup copy job and the seed takes many days)

Thanks,
Aaron

[Vitaliy S.]

Hello Aaron,

Do you have any special reason for limiting the amount of tasks for your repository? What is the current configuration? Do you use WAN accelerators for your backup copy jobs?

Thank you!

[aeccles]

Hello Aaron,

Do you have any special reason for limiting the amount of tasks for your repository? What is the current configuration? Do you use WAN accelerators for your backup copy jobs?

Thank you!

I have the limit on the repository at our remote site. It's accessed via a point to point connection and while it isn't fast, it's sufficient. (We don't own the version that provides WAN acceleration.)

The reason I have it limited to a single job is because I want to prioritize my data protection. If I let all the VMs backup at once, it takes about 12 hours before any of them finish. If there was a disaster at 3am, I would lose all of the data from the previous day because all the jobs would still be running. However, if I limit the repository to a single job and had a disaster at 3am, a few of the servers (the most critical ones) would have finished by then.

Is there anyway to accomplish what I was asking?

[Vitaliy S.]

If you have only 1 VM being sync'ed up with the offsite location by a backup copy job, then you can create only 1 backup copy job and define VM order you need. In this case, all VMs will be processed one by one with your configured priority.

[aeccles]

If you have only 1 VM being sync'ed up with the offsite location by a backup copy job, then you can create only 1 backup copy job and define VM order you need. In this case, all VMs will be processed one by one with your configured priority.

? You lost me... the example I gave in my original post is pretty much how I have things setup. I have multiple backup copy jobs that have multiple vms in them and the order/priority is lost when multiple jobs are waiting for resources.

[Vitaliy S.]

What I was trying to say that, if you want Server 1, Server 2, Server 3 backed up first and then Server A, Server B and C, then why to create two backup copy jobs in the first place? You say that there is a limit on your repository, so that only 1 VM can come through and in order to achieve what you want only 1 backup copy job is required.

As regards your current setup I'm not sure, it is possible to give priority to the first backup copy job and then let the second one to sync up the VMs.

Hope this makes sense.

[aeccles]

Ok, I follow you and I agree with your idea BUT, I have multiple backup copy jobs because of bugs.
#1 I've had previous issues with jobs that have caused all of the backups within the job to require a reseed. By limiting the number of VMs in a job, I'm trying to reduce my exposure to this issue as it takes about a week to seed just 3 of the servers. (This was a long time ago, but it was significant and I learned my lesson.)

#2 I've had previous issues with Backup Copy jobs where the job expires and just hangs - the VMs don't get processed and there are no e-mail alerts. I have to disable and re-enable the backup copy job to fix it. Separating the VMs between jobs reduces my exposure to this issue as well. (Yes, I'm running the latest version and I'm waiting for this to happen again to open a support case)

Those 2 bugs aside, I do think Veeam should provide a way to adhere to the correct order. There are other cases where it could come up (for example, sending backup copy jobs to multiple locations) and cause frustration. The biggest problem I have with it is that I always want my DCs backed up in a specific order and sometimes this issue messes that up. Ultimately, this sounds like unexpected behavior and that inevitably leads to problems.

[Vitaliy S.]

Thanks for clarifications, now I see the reason for this setup. I agree with you that this could potentially be a good functionality to prioritize VMs across multiple jobs, however I would expect all bugs to be fixed if you open a support case and investigate these unexpected behaviors with our technical team. Thanks!

[Gostev]

At 7pm, Job1 kicks off and let say Server2 is still backing up at 8pm... Job 2 now starts and is immediately queued, waiting for resources. When Server 2 finishes, I want Server 3 to start backing up, however Server A backs up next. Other than changing the job times, is there any way to prevent this from happening?

In v8, we've made some changes to intelligent load-balancing, so it will work the way you want it to. Essentially, v8 scheduler will have a priority to finish any started job as soon as possible. If more than one job is running at the same time, they will be prioritized by start time, and backup infrastructure resources will not be assigned to the following jobs if the currently processed job still needs those.

The main driver was that many customers wanted all VMs included in the same job to be backed up as close to each other in time as possible... but there is also a number of other smaller issues that current scheduling approach causes, namely around job retries or periodic jobs.

Basically, v8 intelligent load balancing will be much more intelligent. Above, I am just scratching the surface

[aeccles]

In v8, we've made some changes to intelligent load-balancing, so it will work the way you want it to. Essentially, v8 scheduler will have a priority to finish any started job as soon as possible. If more than one job is running at the same time, they will be prioritized by start time, and backup infrastructure resources will not be assigned to the following jobs if the currently processed job still needs those.

Excellent! Thanks

[dellock6]

v8 scheduler will have a priority to finish any started job as soon as possible. If more than one job is running at the same time, they will be prioritized by start time, and backup infrastructure resources will not be assigned to the following jobs if the currently processed job still needs those.

This is one of the usual little gems under the hood that are for obvious reasons not promoted. But try to have many VMs and many jobs, and you would love this feature like you loved parallel processing. So happy it's coming!