concurrent task handling

[pirx]

Hi,

I can set max concurrent tasks for proxy and for repositories. I'm struggling with this for a while now, but now that I'm setting up new repositories with local storage and servers that will act as proxy and repository server I'm thinking about it again.

Each server has 52 cores and 2 RAID volumes/extents. One volume per server will be a SOBR extent for backup, the other for copy. So I've different scenarios:

- backups are running and using task slots, slots for proxy tasks are needed too
- backups and immediate copy jobs are running at the same time, each es using one server volume, I need slots for backup/copy repo and proxy

I could set repository tasks to 26 for both extents but this would be a waste of resources if either no backup or no copy job would be running. But even if I wouldn't care I'd still need to assign some resources for proxy tasks.

How to I find the right balance? And how do I prevent a proxy from being overloaded? I don't see that there is an overall concurrent task limit that is valid for all types of tasks. Veeams ressource assignment could be a bit more dynamic, for example assign priorities to task types and overall max values.

I tend to just set it to 30/30/30 for the two volumes and proxy tasks. It's still possible that I either do no utilize all cores or worst case overload them with up to 90 tasks.

[foggy]

Right, different components are not aware of how many tasks the others are running, so you should consider the fact that the server performs several roles when configuring the max number of parallel tasks for each of them. Maybe a more creative jobs scheduling throughout a day would help to maximize performance and avoid resource contention.

[pirx]

Yeah, I tried time periods for offload jobs (20:00-08:00 disabled) before but I some jobs were then canceled or stopped as they reached the beginning of this period and had error or warning state. A more adjustable concurrent task handling would be really helpful here.

[foggy]

Jobs chaining could help but we generally don't recommend it due to other side effects it might have.