Feature Request: Option for max number of concurrent jobs

[jyarborough]

I would like to see a configurable option for the maximum number of concurrent backup jobs either globally or per backup file destination. I am comfortable with 2 or 3 jobs running at the same time but I think the maximum is hard set to something like 8. This would make my schedules much more efficient.

As an example, we have 5 sets of backups that could technically fire off at 5pm. If I schedule them all to run at that time and they all start processing then our Veeam Backup server gets killed. I have staggered their schedules to that 3 start at 5pm and then other 2 start at 6:30pm. This should give the 5pm jobs time to complete but if they run over for whatever reason, we might see all 5 jobs running at the same time. On the flip side of that, if the 3 servers finish by say 5:45pm, there is 45 minutes where backups could have been running but weren't because of the staggered schedule.

I've read articles about job chaining, etc, but I really don't want to have to build the chains like that. I think a queue would be ideal for what we are trying to accomplish. In the Backup Exec world, this is exactly how we do all of our jobs. We have numerous ones start at a set time but the destination backup to disk folder only allows 2 current operations so two jobs run and the others are queued. This works very well in that scenario and I would love to see it here also.

Thanks again!

John Yarborough

[Vitaliy S.]

Hello John,

Thank you for taking the time to share your feedback. Some sort of a job repository that is responsible for job queueing makes a good sense. We really appreciate your comments and suggestions!

Thank you!

[Alan_Eden]

Hi Vitaliy,

I'd like to echo this feature request.

We've found that we can reliably run 8 backups concurrently without issues, however attempting to run more than this results in jobs hanging.

We had a incident this week where one of our staff kicked off a further 4 jobs without noticing that there were already 8 backups running at the time. We have 225 backup jobs set, so it can be quite easy to miss a couple of jobs running halfway down the list.

As John suggested, it would be nice if there was a queue for jobs, so that if we exceed a specified concurrent number of jobs, any others that are "started" can be queued so that they start as soon as the number of concurrent jobs drops below the limit. This would be useful to help remove human intervention overloading the server as well as being able to maximise our throughput of data.

Am I correct in thinking that the 8 concurrent job limit in terms of reliability is due to a limitation within the VCB design supporting a maximum number of concurrent connections? If so, does this limit still apply with the vStorage API, or could we, in theory, push the boundaries further?

What is the maximum number of concurrent jobs that has ever been reliably run with Veeam?

Thanks,
Alan

[Gostev]

Alan, there is no known limit - I've seen customers running 14 and more. However, even with iSCSI SAN typically you need no more than 3-4 jobs to fully saturate the backup server. While in case of faster FC SAN, 2 concurrent jobs is enough. There are simply no reason to run more parallel jobs than that, as it will not provide any performance benefit.

So I too agree that queueing jobs is a good idea.