Maintain control of your Microsoft 365 data
Post Reply
aeph
Enthusiast
Posts: 81
Liked: 9 times
Joined: Sep 26, 2024 11:02 am
Contact:

Proxy pool balancing/ scalability question

Post by aeph »

Hi,

We have a proxy pool with three proxies in it. I would be interested to know how the distribution in the pool works, the HelpCenter article is not very informative here (https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=80).

What we have found out:
We have an exchange-only job that runs for a very long time, as this job processes about 3k users (We are in the middle of onboarding, the job has only delivered 2 increments and still has some catching up to do). This job therefore overlaps with other jobs from other organizations with significantly fewer objects. For example, the job of such an organization without a pool only took about 30-50 minutes. In the case that the jobs overlap, such a job suddenly takes 8+h. However, the large Exchange job has very good performance all the time. Wouldn't it make more sense for the Exchange job to decrease a little more performance so that the small organizations can complete the job faster?

What principle is used to distribute the load? First-come = highest priority?
Something we have already observed: A job of a small organization was pushed to a few objects, but the last few objects then took another 2-3 hours.

The utilization of the proxies is around 20-30% CPU and 60-65% RAM.
Sometimes, however, one of the proxies has significantly less r (e.g. 5% CPU and 10% RAM). Looks like the proxies are not the bottleneck.

At the moment we have 2 linux proxies and 1 windows proxy in the pool - Does that make a difference? Should we go exclusively to Linux, for example? Is it better to use several slightly weaker linux proxies than fewer stronger ones, or does that not matter either?

Thank you!
pat_ren
Service Provider
Posts: 94
Liked: 16 times
Joined: Jan 02, 2024 9:13 am
Full Name: Pat
Contact:

Re: Proxy pool balancing/ scalability question

Post by pat_ren »

I'm also interested in this, we have around 550 repos/550 jobs, and I have 8 windows proxy with 8vcpu/32gb RAM each in a proxy pool and I find that the CPU usage is usually 20-40% max and seems like the proxy threads count could be a lot higher on each one (it used to get close to 100% in v7 when we could tune the proxy load better) and the jobs now take a lot longer to run than they did before when each job was using a specific repo. I'm not sure if I should if it would be better to have more proxy's with less resources, or less proxys with more resources. So far the proxy pool seems slower and less efficient than in v7.
edh
Veeam Legend
Posts: 368
Liked: 114 times
Joined: Nov 02, 2020 2:48 pm
Full Name: Manuel Rios
Location: Madrid, Spain
Contact:

Re: Proxy pool balancing/ scalability question

Post by edh »

In one post, they said that 1 Core = 4 Task item.
In our case we got 80 HT with a task limit of 320 items at time. And we're able to see 95-98% of CPU usage and 40GB of 64GB RAM used, and windows as OS.
We have the sense that linux proxy will be more sensitive to .net changes than Windows and you should block the patches. I think that actually Veeam priority system is a FIFO , doesnt care if backup task is full or increment. Just first-in-frist out.
Service Provider | VMCE
Post Reply

Who is online

Users browsing this forum: sfey and 84 guests