Enhancement request -- job priority

[ChuckS42]

The application running on these servers doesn't like the stun a snapshot creates, so they set aside a backup window during specific hours to allow a backup to run when the stun can be tolerated. Outside this window a stun will interrupt critical processes on the servers.

I see what you're saying with regards to backup windows, but that seems to be an awful complicated way to work around a simple problem. I could stand up dedicated proxy and repository servers just for this job, but that's an overly complicated solution as well.

A simple high-priority flag (for an individual VM in a job, or for the entire job) would make this much easier.

[foggy]

I see the point, you need to make sure the job gets resources once it starts.

[Tarqy]

Apologies for bumping an old thread but are there any plans to implement this in a future version?

I have the same issue as Chuck above, I have a some servers that runs various maintenance and batch task overnight and there is only a short period of time where we can do the stun for backup but we are finding that starting the job at this time is not sufficient to guarantee access to resources.

Thanks

[foggy]

Hi Barry, cannot say anything regarding the plans, but in addition to the workaround mentioned above, you can set up a dedicated proxy server assigned exclusively for this job.

[Tarqy]

This is a Hyper-V host using on-host processing, the resource contention is on the repository and having moved to a SOBR having to add in a dedicated repository for this particular jobs seems like going backwards!

[foggy]

Ok, fair enough. Thanks for your feedback.

[meeyou]

Just piling on to add another request for this feature.

We have a select few jobs that are very time sensitive (the server has to be powered off for the job to complete due to DirectPath I/O device passed through) so we need this one job to cut in line and take the very next available resource slots when it starts, otherwise it is doomed for failure after its window.

[Gostev]

Thanks for sharing your use case, John.

[ddubois]

Back Story: I have quite a number of chained backup jobs and three backup copy jobs. When the backup copy jobs are transferring data, they grab many concurrent tasks from the repository. If the repository is set too low, these jobs will hold up the normal backup jobs. The problem is that two of the backup copy jobs go to an offsite location and some of the larger VMs can hold up the backups for hours. I have the concurrent tasks on the Proxy set to 1 as I just want sequential writing to the repository for deduplication purposes. There is no current way to limit the backup copy jobs to x number of concurrent tasks. They do not use a proxy and only have the shared resource of the repository to limit the tasks! The backup copy jobs NEED an in-job control of the number of concurrent tasks. This is my main issue. I do have one other issue I am dealing with though that I will mention in case someone has a good suggestion for it. My two jobs that copy backups offsite run on different schedules. One job has a daily cycle for the more critical business applications. The second remote copy job has an interval set to one week to pick up the weeklies of the less changing VMs. I would like to limit each of these jobs to one concurrent task so that they might get equal bandwidth but I would also love to be able to group these two jobs and set a priority on them so that one runs before the other. A weighting system maybe. I would like the ones on the daily schedule to complete before it tries to copy the weekly job if that makes sense...

Any suggestions would be appreciated but definitely would love to see a concurrent task limit on backup copy jobs!
Case# 02338963

[tsightler]

Perhaps I'm not fully grasping the issue, but if you just want to limit the concurrency of the backup copy job you can set the task limit of the target repository.

[ddubois]

I have two jobs going to the remote repository, a daily copy job and a weekly interval job. If I set the destination repository to say 2, then one of the jobs will grab both unfortunately and make the other wait and there is no way to guarantee which job will grab them. Unfortunately, right now, I am looking at the two jobs running, and the weekly job is using about 95% of the bandwidth and the daily job is using the remaining 5%. I wish I could A) control the concurrent tasks on each job individually, and B) set a priority so that the daily job will take precedence and complete instead of it being a crap shoot as to which gets the most bandwidth and number of tasks. I mean the daily has only 24 hours to complete, but the weekly job can take up to 7 days. At this rate, my daily will not complete within its interval and the weekly will finish in less than 2 days. Hopefully that illustrates the issue better. One remote job has 3 tasks running and the other has 1 (luckily).

[ddubois]

I thought about placing all of the VMs into one remote copy job and tried it. Because I keep daily backups on the VMs that I want to copy on a weekly schedule, it wants to sync those VMs daily as well as the priority ones that I backup every 2 hours. With it trying to keep up with the changes, it will not complete in the allotted time and errors out. So it became two jobs with different intervals. However, now I cannot control them as I need to be able to.

[foggy]

Hi Derek, I'm merging the thread since your request comes down to the job priority feature mostly. Thanks for the feedback.

[ddubois]

And a way to limit tasks on backup copy jobs going to the same repository... This is very frustrating as I cannot figure out a way to get my daily offsite jobs to complete without the weekly job getting in the way. We are great as far as local jobs and retention policies but if anything ever happens to this place, we are in trouble. I was hoping that there would be a workaround. Wow, this has been open for a long time. There have to be many others in my same situation. What is everyone else doing to get around it?

[ddubois]

Instead of two separate backup copy jobs, I tried one, but then the systems that I want to copy offsite do not complete in time. I would like some of the systems to copy daily and the rest copy weekly however since all of those backup locally at least once a day, the ones that I want weekly of at the remote site, copy every day with the others causing too much traffic to complete. With the two copy jobs, one on a daily cycle and the other on a weekly cycle, it seems the weekly occupies most of the bandwidth causing the daily job to not complete for the first half of the week. If only I could get veeam to put a preference on the daily job over the weekly one, my problems would be over. I am desperately seeking a work around until the software can do that.

[ddubois]

It has been almost another year so I wanted to bump this to see if there were any enhancements planned for a priority option yet. I still battle getting backups to my offsite repository. One daily backup copy job and one weekly. I need the daily to take priority over the weekly due to bandwidth. The weekly finishes in the first half of the week while the daily fails for the first half then works for the remainder of the days. I have been looking at other solutions to accomplish this but would really love to be able to use Veeam for everything. Need a way to tell Veeam that if other jobs are running, pause them and run this high priority job first.

[veremin]

Currently, we can only say that this feature is not on the short-term roadmap.

But have you tried to set a start time for weekly copy job a bit earlier,so that, it starts sooner than daily one does, occupies all available resources and forces the daily job to wait?

Thanks.

[zlewis]

I was noticing that on occasion, I have clients who will do a deployment of a new software or or update and that will cause backups and backup copies to take longer than usual. This causes offsite copies to fail for a day or two.

I would like to suggest Priority Levels in the job itself.
Essentially, have a default priority (defined by the user as a number) that all new systems would fall under. Then have the ability to override the default priority for each system with a higher or lower number to manually adjust what gets backed up. Then, when the job is running, it would search for systems with the lowest number first and do those backups/copies first. Once they are done, move to the next number.
That way, if you want to define that your primary domain controller has a higher priority, but the secondary domain controller has a lower priority, Veeam is not attempting to back up both of them at the same time while the business critical app server or file server is towards the end of the list and gets skipped because time ran out.

[foggy]

Hi Zack, currently VMs are processed by the job in the order they are listed in the job settings. So you can arrange them in the required order, will that address your request?

[matthewingram]

I agree. We have a dozen or so jobs. I would like to have a priority option between running jobs. If two jobs are running at the same time, say production and non-production i'd like to give priority to production. That's just one example.

[foggy]

Ok, then I'm merging your request to the corresponding thread.

[ChuckS42]

Hi Zack, currently VMs are processed by the job in the order they are listed in the job settings. So you can arrange them in the required order, will that address your request?

If you're building backup jobs based on tags or folders, this won't work.

[foggy]

Can you create tags based on VMs criticality to address this?

[mrholm]

Hi
Is there any discussions internally at Veeam around adding the possibilty to set priority levels on backup jobs.
We have one of our main Veeam sites have over 80 jobs runinng each night. Several of these are jobs on test and development servers.
We still want to start the majority of the jobs at same time but then we can't full sure say that the production jobs start backup before the test servers.
So if possible a possibility to set different level of priority on jobs and have Veeam have queued jobs and processes run based on priority.

//Mats

[foggy]

Hi Mats, cannot comment on the internal discussions but merging you into the existing forum one. Currently, you can achieve the prioritization by starting the more important jobs first, with a short gap ahead of the less important ones - this would ensure they are queued appropriately. Anyway, thanks for the feedback.

[PetrM]

One more idea is to use "after this job" option to run low priority jobs after the production one.

Thanks!

[Gostev]

We're adding the ability to tag some jobs as high priority in v11. Such jobs will have priority in obtaining backup infrastructure resources from the scheduler, comparing to regular jobs. Thanks!