Enhancement request -- job priority

Availability for the Always-On Enterprise

Enhancement request -- job priority

Veeam Logoby averylarry » Wed Sep 04, 2013 1:42 am

At least I couldn't find any way to do this.

I have some local backups, some local replications, and some WAN replications that run each night. With v.7 I've turned on parallel processing. What I am experiencing is the local replication and local backup jobs "eating up" all the parallel processing "slots". This means my WAN replications jobs will at times sit idle while the other jobs are running multiple VMs.

So I'd like to request a job priority feature. I could put my WAN replications jobs on a higher priority than my local jobs. That way the WAN replications jobs would always be using at least some of the parallel processing "slots" available (except for the normal conflicts). I suppose I could disable the parallel processing, but what fun would that be?


On a side note -- when a VM finishes processing, how does Veeam decide which VM in which job gets processed next (after eliminating VMs that have a conflict)?
averylarry
Expert
 
Posts: 258
Liked: 28 times
Joined: Tue Mar 22, 2011 7:43 pm
Full Name: Ted

Re: Enhancement request -- job priority

Veeam Logoby averylarry » Wed Sep 04, 2013 1:45 am

I'll do some experimenting. I think it might process the jobs in alphabetical order.
averylarry
Expert
 
Posts: 258
Liked: 28 times
Joined: Tue Mar 22, 2011 7:43 pm
Full Name: Ted

Re: Enhancement request -- job priority

Veeam Logoby Gostev » Wed Sep 04, 2013 2:34 am

Jobs are processed according to their start times. Make sure your high priority jobs are set to start earlier than other.

I don't disagree job priorities can still be helpful in certain scenarios. We will look at v7 in the wild, and decide how to best implement those. Meanwhile, issues like above can be easily resolved by deploying more proxies, or reducing max concurrent ingress tasks on repositories, or just using dedicated proxy sets for all your jobs (or any combination of these approaches).
Gostev
Veeam Software
 
Posts: 21510
Liked: 2381 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Enhancement request -- job priority

Veeam Logoby dellock6 » Wed Sep 04, 2013 12:41 pm

Also, be careful about VMs with several disks, since a single VM with let's say 5 disk is going to use 5 processing slots. You may want to place the jobs with those VMs to be started after others, but if for example a VM fails, is placed in the retry queue, so somehow your schedule is not going to be fully followed by Veeam. Unless you configure retries differently.

I'm in the phase of tuning our backup infrastructure after a couple of weeks using V7, right because of these kind of findings.

Luca.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5118
Liked: 1361 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Enhancement request -- job priority

Veeam Logoby averylarry » Fri Sep 06, 2013 2:55 pm

Gostev --

Jobs begin based on their start times, sure. But I end up with 5-10 jobs running all at the same time. They are not processed in series based on their start times. They are processed in parallel. Now with the additional feature of parallel processing, I want to know how the Veeam server decides which VM in which job to process when a VM in whatever job finishes.

Also -- I'm a small business. Cannot afford more proxies (I can't even afford a dedicated proxy). I don't understand how the other suggestions would make any difference.


Luca --

Note that hotadd mode only does 1 disk at a time, so parallel processing is strictly for multiple simultaneous VMs.


Ultimately I want the ability to make sure my WAN is always being used to avoid running into the work day. A dedicated WAN replication proxy would accomplish this (and a good idea I didn't consider). I just don't consider that feasible for many small businesses (I have minimized the number of proxies I use because they are production machines and making them a proxy will break hotadd mode and direct SAN mode also costs more money in infrastructure and network mode is generally slow/inefficient).
averylarry
Expert
 
Posts: 258
Liked: 28 times
Joined: Tue Mar 22, 2011 7:43 pm
Full Name: Ted

Re: Enhancement request -- job priority

Veeam Logoby foggy » Fri Sep 06, 2013 3:06 pm

averylarry wrote:Now with the additional feature of parallel processing, I want to know how the Veeam server decides which VM in which job to process when a VM in whatever job finishes.

This thread is likely to answer your question.
foggy
Veeam Software
 
Posts: 15087
Liked: 1110 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Enhancement request -- job priority

Veeam Logoby JeremiahS » Sat Sep 07, 2013 1:33 pm

You may want to try chaining jobs to avoid resource contention and allow manual job prioritization. Under schedule, configure highest priority job to at desired time with next highest priority time starting after job1 finishes and so on. With limited proxies and single repository design, this approach seems to be working reasonably well.
JeremiahS
Novice
 
Posts: 4
Liked: never
Joined: Mon Aug 19, 2013 6:23 pm
Full Name: Jeremiah S

Re: Enhancement request -- job priority

Veeam Logoby averylarry » Thu Sep 12, 2013 7:13 pm

Jeremiah --

Thanks for the idea. The problem is I can come up with a dozen different workarounds. I would just like something that I would consider more of a "fix" to directly address an issue I've found, instead of figuring out a way to circumvent (what I consider) a limitation.
averylarry
Expert
 
Posts: 258
Liked: 28 times
Joined: Tue Mar 22, 2011 7:43 pm
Full Name: Ted

Re: Enhancement request -- job priority

Veeam Logoby larry » Thu Sep 12, 2013 7:26 pm

I would like this as well.
I found with backupcopy jobs at times will do 3 VMs from one job, a VM from job 3 then back to job 1 for a vm so alot of jobs endup open and running. The jobs are continuos but the backup job I start in the order I want them to run.
larry
Expert
 
Posts: 380
Liked: 89 times
Joined: Wed Mar 24, 2010 5:47 pm
Full Name: Larry Walker

Re: Enhancement request -- job priority

Veeam Logoby v.Eremin » Fri Sep 13, 2013 7:54 am

Hi, Larry. You can add particular VMs, instead of backup jobs, as a source for backup copy job and explicitly set the processing order, using “Up” and “Down” buttons. Thanks.
v.Eremin
Veeam Software
 
Posts: 13558
Liked: 1006 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Enhancement request -- job priority

Veeam Logoby larry » Fri Sep 13, 2013 7:03 pm

I like the add by job too much. With 7 all my backup jobs have 2 backupcopy jobs to send to DR and tape, same name with -DR added. This way once you place a VM into a backupjob the backup to tape and offsite is automatic. Before we would get a vm not going to tape or offsite but getting backed up, this way it will just happen. Our jobs are VM A-C ( all vms who names start with A-C, VM D-G and so on. We name the backup location, tape job and DR spot the same. We do pull out a couple servers that get treated special, exchange and SQL. With one quick look you know where the VM should be as well as its offsite copy. thats my reasoning.
larry
Expert
 
Posts: 380
Liked: 89 times
Joined: Wed Mar 24, 2010 5:47 pm
Full Name: Larry Walker

[MERGED] Job Priorities

Veeam Logoby ekisner » Tue Dec 03, 2013 5:06 pm

I'm presently in a situation where I've got my replication jobs disabled... going to be doing some vcenter maintenance "at some point". Since some jobs can take a long time, and I'd rather not take vcenter offline while there's a job running, disable seems like the logical choice.

I've just pushed back the maintenance however, and re-enabled the jobs in the short term. Since they were disabled and out of date I initiated a manual run to get everything current. And since I have jobs on a 1:1 basis with VMs, I now have quite a few active jobs.

I could see many other situations where this would be applicable too, but that's how I came to the thought that I'm now suggesting.

It would be nice to be able to prioritize resources to jobs.. the "more important" jobs can hence finish sooner, and the "less important jobs" can finish whenever.
ekisner
Expert
 
Posts: 141
Liked: 31 times
Joined: Thu Jul 26, 2012 8:04 pm
Full Name: Erik Kisner

Re: Enhancement request -- job priority

Veeam Logoby foggy » Wed Dec 04, 2013 6:38 am

Thanks for your feedback. Currently you can prioritize jobs according to their start time (the job that started first will be processed first).
foggy
Veeam Software
 
Posts: 15087
Liked: 1110 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED] Job Priorities

Veeam Logoby bsoosay » Tue Oct 14, 2014 12:05 am

I've configured my Backups by datastores. I have about 35 datastores.

I have 6 proxy servers with 4 cores, which totals up to 24 connections to the repository.

Now when I execute all 35 jobs, how does Veeam prioritise which job runs first?

1. Is it by alphabetical order, sequential order or some other method?

2. Is there a way for me to prioritise which job runs first?
bsoosay
Enthusiast
 
Posts: 34
Liked: never
Joined: Thu Oct 02, 2014 7:47 am
Full Name: Benjamin Soosay

Re: Job Priorities

Veeam Logoby foggy » Tue Oct 14, 2014 8:47 am

Jobs are executed right in the order you start them (literally click Start on the job) and tasks are assigned to proxy servers upon availability of processing slots.
foggy
Veeam Software
 
Posts: 15087
Liked: 1110 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: No registered users and 31 guests