Comprehensive data protection for all workloads
ChuckS42
Expert
Posts: 189
Liked: 27 times
Joined: Apr 24, 2013 8:53 pm
Full Name: Chuck Stevens
Location: Seattle, WA
Contact:

Re: Enhancement request -- job priority

Post by ChuckS42 »

The application running on these servers doesn't like the stun a snapshot creates, so they set aside a backup window during specific hours to allow a backup to run when the stun can be tolerated. Outside this window a stun will interrupt critical processes on the servers.

I see what you're saying with regards to backup windows, but that seems to be an awful complicated way to work around a simple problem. I could stand up dedicated proxy and repository servers just for this job, but that's an overly complicated solution as well.

A simple high-priority flag (for an individual VM in a job, or for the entire job) would make this much easier.
Veeaming since 2013
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

I see the point, you need to make sure the job gets resources once it starts.
Tarqy
Service Provider
Posts: 42
Liked: 5 times
Joined: Aug 08, 2014 1:51 pm
Full Name: Barry Knox
Contact:

Re: Enhancement request -- job priority

Post by Tarqy »

Apologies for bumping an old thread but are there any plans to implement this in a future version?

I have the same issue as Chuck above, I have a some servers that runs various maintenance and batch task overnight and there is only a short period of time where we can do the stun for backup but we are finding that starting the job at this time is not sufficient to guarantee access to resources.

Thanks
VMCE
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

Hi Barry, cannot say anything regarding the plans, but in addition to the workaround mentioned above, you can set up a dedicated proxy server assigned exclusively for this job.
Tarqy
Service Provider
Posts: 42
Liked: 5 times
Joined: Aug 08, 2014 1:51 pm
Full Name: Barry Knox
Contact:

Re: Enhancement request -- job priority

Post by Tarqy »

This is a Hyper-V host using on-host processing, the resource contention is on the repository and having moved to a SOBR having to add in a dedicated repository for this particular jobs seems like going backwards!
VMCE
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

Ok, fair enough. Thanks for your feedback.
meeyou
Influencer
Posts: 22
Liked: never
Joined: Oct 16, 2012 5:47 pm
Full Name: John White
Contact:

Re: Enhancement request -- job priority

Post by meeyou »

Just piling on to add another request for this feature.

We have a select few jobs that are very time sensitive (the server has to be powered off for the job to complete due to DirectPath I/O device passed through) so we need this one job to cut in line and take the very next available resource slots when it starts, otherwise it is doomed for failure after its window.
Gostev
Chief Product Officer
Posts: 31773
Liked: 6942 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Enhancement request -- job priority

Post by Gostev »

Thanks for sharing your use case, John.
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

[MERGED] [Feature Request] Concurrent Backup Copy Job Tasks

Post by ddubois »

Back Story: I have quite a number of chained backup jobs and three backup copy jobs. When the backup copy jobs are transferring data, they grab many concurrent tasks from the repository. If the repository is set too low, these jobs will hold up the normal backup jobs. The problem is that two of the backup copy jobs go to an offsite location and some of the larger VMs can hold up the backups for hours. I have the concurrent tasks on the Proxy set to 1 as I just want sequential writing to the repository for deduplication purposes. There is no current way to limit the backup copy jobs to x number of concurrent tasks. They do not use a proxy and only have the shared resource of the repository to limit the tasks! The backup copy jobs NEED an in-job control of the number of concurrent tasks. This is my main issue. I do have one other issue I am dealing with though that I will mention in case someone has a good suggestion for it. My two jobs that copy backups offsite run on different schedules. One job has a daily cycle for the more critical business applications. The second remote copy job has an interval set to one week to pick up the weeklies of the less changing VMs. I would like to limit each of these jobs to one concurrent task so that they might get equal bandwidth but I would also love to be able to group these two jobs and set a priority on them so that one runs before the other. A weighting system maybe. I would like the ones on the daily schedule to complete before it tries to copy the weekly job if that makes sense...

Any suggestions would be appreciated but definitely would love to see a concurrent task limit on backup copy jobs!
Case# 02338963
tsightler
VP, Product Management
Posts: 6018
Liked: 2848 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: [Feature Request] Concurrent Backup Copy Job Tasks

Post by tsightler »

Perhaps I'm not fully grasping the issue, but if you just want to limit the concurrency of the backup copy job you can set the task limit of the target repository.
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

Re: [Feature Request] Concurrent Backup Copy Job Tasks

Post by ddubois »

I have two jobs going to the remote repository, a daily copy job and a weekly interval job. If I set the destination repository to say 2, then one of the jobs will grab both unfortunately and make the other wait and there is no way to guarantee which job will grab them. Unfortunately, right now, I am looking at the two jobs running, and the weekly job is using about 95% of the bandwidth and the daily job is using the remaining 5%. I wish I could A) control the concurrent tasks on each job individually, and B) set a priority so that the daily job will take precedence and complete instead of it being a crap shoot as to which gets the most bandwidth and number of tasks. I mean the daily has only 24 hours to complete, but the weekly job can take up to 7 days. At this rate, my daily will not complete within its interval and the weekly will finish in less than 2 days. Hopefully that illustrates the issue better. One remote job has 3 tasks running and the other has 1 (luckily).
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

Re: [Feature Request] Concurrent Backup Copy Job Tasks

Post by ddubois »

I thought about placing all of the VMs into one remote copy job and tried it. Because I keep daily backups on the VMs that I want to copy on a weekly schedule, it wants to sync those VMs daily as well as the priority ones that I backup every 2 hours. With it trying to keep up with the changes, it will not complete in the allotted time and errors out. So it became two jobs with different intervals. However, now I cannot control them as I need to be able to.
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

Hi Derek, I'm merging the thread since your request comes down to the job priority feature mostly. Thanks for the feedback.
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

Re: Enhancement request -- job priority

Post by ddubois »

And a way to limit tasks on backup copy jobs going to the same repository... This is very frustrating as I cannot figure out a way to get my daily offsite jobs to complete without the weekly job getting in the way. We are great as far as local jobs and retention policies but if anything ever happens to this place, we are in trouble. I was hoping that there would be a workaround. Wow, this has been open for a long time. There have to be many others in my same situation. What is everyone else doing to get around it?
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

Re: Enhancement request -- job priority

Post by ddubois »

Instead of two separate backup copy jobs, I tried one, but then the systems that I want to copy offsite do not complete in time. I would like some of the systems to copy daily and the rest copy weekly however since all of those backup locally at least once a day, the ones that I want weekly of at the remote site, copy every day with the others causing too much traffic to complete. With the two copy jobs, one on a daily cycle and the other on a weekly cycle, it seems the weekly occupies most of the bandwidth causing the daily job to not complete for the first half of the week. If only I could get veeam to put a preference on the daily job over the weekly one, my problems would be over. I am desperately seeking a work around until the software can do that.
ddubois
Influencer
Posts: 14
Liked: never
Joined: Oct 11, 2017 5:57 pm
Full Name: Derek DuBois
Contact:

Re: Enhancement request -- job priority

Post by ddubois »

It has been almost another year so I wanted to bump this to see if there were any enhancements planned for a priority option yet. I still battle getting backups to my offsite repository. One daily backup copy job and one weekly. I need the daily to take priority over the weekly due to bandwidth. The weekly finishes in the first half of the week while the daily fails for the first half then works for the remainder of the days. I have been looking at other solutions to accomplish this but would really love to be able to use Veeam for everything. Need a way to tell Veeam that if other jobs are running, pause them and run this high priority job first.
veremin
Product Manager
Posts: 20291
Liked: 2259 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Enhancement request -- job priority

Post by veremin »

Currently, we can only say that this feature is not on the short-term roadmap.

But have you tried to set a start time for weekly copy job a bit earlier,so that, it starts sooner than daily one does, occupies all available resources and forces the daily job to wait?

Thanks.
zlewis
Service Provider
Posts: 1
Liked: never
Joined: Mar 25, 2019 9:37 pm
Full Name: Zack Lewis
Location: Jefferson City, MO
Contact:

[MERGED] Feature Request: Backup Priority Levels

Post by zlewis »

I was noticing that on occasion, I have clients who will do a deployment of a new software or or update and that will cause backups and backup copies to take longer than usual. This causes offsite copies to fail for a day or two.

I would like to suggest Priority Levels in the job itself.
Essentially, have a default priority (defined by the user as a number) that all new systems would fall under. Then have the ability to override the default priority for each system with a higher or lower number to manually adjust what gets backed up. Then, when the job is running, it would search for systems with the lowest number first and do those backups/copies first. Once they are done, move to the next number.
That way, if you want to define that your primary domain controller has a higher priority, but the secondary domain controller has a lower priority, Veeam is not attempting to back up both of them at the same time while the business critical app server or file server is towards the end of the list and gets skipped because time ran out.
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature Request: Backup Priority Levels

Post by foggy »

Hi Zack, currently VMs are processed by the job in the order they are listed in the job settings. So you can arrange them in the required order, will that address your request?
matthewingram
Influencer
Posts: 20
Liked: never
Joined: Oct 03, 2018 10:13 pm
Contact:

Re: Feature Request: Backup Priority Levels

Post by matthewingram »

I agree. We have a dozen or so jobs. I would like to have a priority option between running jobs. If two jobs are running at the same time, say production and non-production i'd like to give priority to production. That's just one example.
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature Request: Backup Priority Levels

Post by foggy »

Ok, then I'm merging your request to the corresponding thread.
ChuckS42
Expert
Posts: 189
Liked: 27 times
Joined: Apr 24, 2013 8:53 pm
Full Name: Chuck Stevens
Location: Seattle, WA
Contact:

Re: Feature Request: Backup Priority Levels

Post by ChuckS42 »

foggy wrote: May 21, 2019 5:39 pm Hi Zack, currently VMs are processed by the job in the order they are listed in the job settings. So you can arrange them in the required order, will that address your request?
If you're building backup jobs based on tags or folders, this won't work.
Veeaming since 2013
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

Can you create tags based on VMs criticality to address this?
mrholm
Expert
Posts: 170
Liked: 15 times
Joined: Apr 20, 2018 8:12 am
Full Name: Mats Holm
Contact:

[MERGED] Priority on backup jobs?

Post by mrholm »

Hi
Is there any discussions internally at Veeam around adding the possibilty to set priority levels on backup jobs.
We have one of our main Veeam sites have over 80 jobs runinng each night. Several of these are jobs on test and development servers.
We still want to start the majority of the jobs at same time but then we can't full sure say that the production jobs start backup before the test servers.
So if possible a possibility to set different level of priority on jobs and have Veeam have queued jobs and processes run based on priority.

//Mats
foggy
Veeam Software
Posts: 21093
Liked: 2123 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Enhancement request -- job priority

Post by foggy »

Hi Mats, cannot comment on the internal discussions but merging you into the existing forum one. Currently, you can achieve the prioritization by starting the more important jobs first, with a short gap ahead of the less important ones - this would ensure they are queued appropriately. Anyway, thanks for the feedback.
PetrM
Veeam Software
Posts: 3393
Liked: 558 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Enhancement request -- job priority

Post by PetrM »

One more idea is to use "after this job" option to run low priority jobs after the production one.

Thanks!
Gostev
Chief Product Officer
Posts: 31773
Liked: 6942 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Enhancement request -- job priority

Post by Gostev » 2 people like this post

We're adding the ability to tag some jobs as high priority in v11. Such jobs will have priority in obtaining backup infrastructure resources from the scheduler, comparing to regular jobs. Thanks!
Post Reply

Who is online

Users browsing this forum: tyler.jurgens and 145 guests