Comprehensive data protection for all workloads
Post Reply
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

(stupid?) idea/suggestion for optimizing job queue

Post by hyvokar »

Hi!

Every now and then it happens so, that the last job/disk in your backup job is the biggest one. And if you have more disks than you have cpus/cores, it might just start after everyting else is finished.
Bored staring at the job process window, I started thinking why not optimize the job queue inside the backup job?

Let's presume I have 4cores on my proxy and I can handle 4 concurrent jobs.

("o" represents for example 10GB of data / 1min, [nxo] represents a vdisk)


"abcd"/"random" job queue could sometimes be like this :(
cpu1: [ooooo][ooooo][ooooo][ooooo][oooooooooooooooooooo] 40min
cpu2: [ooooo][ooooo][ooooo][ooooo][ooooooooooooooo] 30min
cpu3: [ooooo][ooooo][ooooo][ooooo] 20min
cpu4: [ooooo][ooooo][ooooo][ooooo] 20min
:: total time for job 40min


"biggest first" job queue:
cpu1: [oooooooooooooooooooo ][ooooo][ooooo] 30min
cpu2: [ooooooooooooooo ][ooooo][ooooo][ooooo] 30min
cpu3: [ooooo][ooooo][ooooo][ooooo][ooooo][ooooo] 30min
cpu4: [ooooo][ooooo][ooooo][ooooo][ooooo] 25min
:: total time for job 30min


So optimizing jobs queue might just save a some time. This example is for the first time active full backup, but I guess it could also be useful on incremental backups (read the biggest changes first).
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
hyvokar
Veteran
Posts: 411
Liked: 31 times
Joined: Nov 21, 2014 10:05 pm
Contact:

Re: (stupid?) idea/suggestion for optimizing job queue

Post by hyvokar » 1 person likes this post

There is an option for manual sorting on "virtual machines" tab. However, that only let's you sort the VMs, not disks (ok, the biggest VM propably has the biggest disks, but not neccessarily). I would like to have a checkbox "biggest disks first" and possibly "biggest VMs first" (if you want to process one VM at the time). Or something similar :) Autoarrange could also be useful when you add 100VMs and dont want to sort them manually.
Bed?! Beds for sleepy people! Lets get a kebab and go to a disco!
MS MCSA, MCITP, MCTS, MCP
VMWare VCP5-DCV
Veeam VMCE
Gostev
Chief Product Officer
Posts: 31814
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: (stupid?) idea/suggestion for optimizing job queue

Post by Gostev »

Hi. Initial reaction - this is hardly doable since it is impossible to predict the time it will take to process a given VM before the snapshot is taken, and we can query CBT and MFT. For example, a full backup of a VM with five 2 TB disk can be processed in just a few minutes, if they are mostly empty. Thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 78 guests