[Feature Request] - WAN Accelerator priority on repository

[negrumanuel]

Hi all,

First post here - apologies if it's in the wrong location - please move.

WAN acc svc seems to be much slower at "getting a slot" at repository level. In a scenarios where a repo is capped at a low x concurrent tasks, if there are backup health checks and incremental merges into full, those would always take precedence over the WAN acc disk copying.
I would like to be able to either:

- have a repo level toggle setting to say "don't count the WAN acc svc task in the total concurrent tasks". So WAN acc svc will always have a "dedicated line/slot" at repo level. The service is anyway single disk threaded - so can anyway process 1 disk at the time - so it's not like it will overload the repo. The concurrent tasks settings would only apply then to remaining tasks/activities (backuping, merging, health checks, etc).

OR

- be able to increase the WAN acc svc priority via a setting or reg key so that it take precedence over other tasks/activities.

Let me know your thoughts.
Thank you
Manuel.

[david.domask]

Hi Manuel, welcome to the forums.

Can you please clarify, you're discussing WAN Accelerators when you write WAN acc svc? I will adjust the topic title accordingly as it's a bit unclear.

And can you clarify a bit more on the issue you're facing? Was there a Support case that investigated this behavior?

If it's about WAN Accelerators, the tasks get allocated based on the job type, regardless of whether you use WAN Accelerators or not. You are correct that the disk processing with WAN Accelerators is sequential, but my guess is that what you're seeing is the job priority taking place and if the source jobs had to perform a synthetic operation (synthetic full, merge) OR Health Check, that will indeed take priority over the Backup Copy.

But please clarify the behavior you're seeing and share a case number if you already had one to review the behavior.

[negrumanuel]

Hi David,

Indeed, WAN Accelerator - apologies, went for the short version. There was a case logged yesterday which pointed me to raise a feature request here. Case # 08186960

I gave an example in the ticket (I hope you can see it) - not sure if to paste it here to make this too long. But in a nutshell what I am seeing is the following:

- Multiple Agent based backups jobs (with 2-6 computers per job) set to run throughout the night. At xx interval apart.
- Each backup job has its own copy job set to execute immediately when the backup file hits the repository.
- Copy jobs copy the backups from the onprem repo, via the WAN accelerator pair, to an Azure file Share.
- On-prem storage repo is capped at 3 concurrent tasks.

- Backup job B1 would start & finish.
- Copy job C1 would start immediately after. Job consumes 1 out of the 3 concurrent repo tasks.
- Backup job B2 would start some time later. B2 has to perform both merging into full and a backup health check - so takes considerably longer.
- Backup B2 has to process 5 computers lets say. It starts processing first 2 (cause we only have 2 available concurrent repo tasks available - one is held by the copy job) and queues the remaining.
- Copy job C1 finishes processing the disk it was locked onto - in theory, now it should start the next disk. The concurrent repo task slot is released.
- Backup B2 fetches quickly that slot and it allocates to next in the queue. And it keeps doing that until all computers are processed.
- Only when one concurrent task slot is freed up, the copy job C1 can resume disk processing / copying.

But if I have another backup job B3 queued some time after - the effect is then prolonged. My copy job will only be able to catch onto a concurrency repo slot only when there are no backup tasks going on or the backup jobs tasks is -1 my concurrency limit.

Either make an option to not count the copy job slots in the overall repo concurrency slots or allow me to put the copy jobs higher up on the priority ladder so that they stand a chance to grab onto a repo concurrency slot.

I hope this helps explain the behavior/picture I am seeing ?

Thank you!

[david.domask]

Yep, much more clear; the confusion came because WAN Accelerators don't have any tasks themselves, they're engaged as part of Replica / Backup Copy jobs, so the task contention comes from the job tasks, not from the WAN Accelerator itself.

I suppose the main request here is to allow setting specific priorities on jobs regardless of type. Currently Backup Copies always have lower priority.

You can consider removing the Health Check and using a SureBackup job to perform the Content Scan & Backup Verification to help reduce the time the primary job takes.

We cannot omit Backup Copy tasks from the concurrent task scheduling, Backup Copies are not simple file copies, so their resource consumption needs to be planned and considered by the scheduler like any other job, but we can discuss allowing setting custom priority for specific jobs.

[negrumanuel]

I see. I assume there is no mechanism to alter currently priorities between jobs to bump up Copy jobs higher ?

[david.domask]

Correct, for backup copies this functionality does not exist currently, and that is what will be discussed.

Similarly, if it's possible to adjust your architecture (more tasks on repository / WAN accel server or move WAN accel role to another local server), that may help your situation, but understood that hardware constraints are very real for the foreseeable future everywhere

[negrumanuel]

Thanks David for hopefully including this topic somewhere in an internal conversation. Ultimately, I just want a bit more priority for the WAN accelerator copy jobs and also for the service itself to get more oomph. It's grinding to a near halt when a health check operation is running on the same server. (server acts as backup repository with ISCSI volumes, WAN source, etc). (and it's keeping busy one of those repo concurrency threads)
This svc needs to be made first class citizen.