Hi,
We have a proxy pool with three proxies in it. I would be interested to know how the distribution in the pool works, the HelpCenter article is not very informative here (https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=80).
What we have found out:
We have an exchange-only job that runs for a very long time, as this job processes about 3k users (We are in the middle of onboarding, the job has only delivered 2 increments and still has some catching up to do). This job therefore overlaps with other jobs from other organizations with significantly fewer objects. For example, the job of such an organization without a pool only took about 30-50 minutes. In the case that the jobs overlap, such a job suddenly takes 8+h. However, the large Exchange job has very good performance all the time. Wouldn't it make more sense for the Exchange job to decrease a little more performance so that the small organizations can complete the job faster?
What principle is used to distribute the load? First-come = highest priority?
Something we have already observed: A job of a small organization was pushed to a few objects, but the last few objects then took another 2-3 hours.
The utilization of the proxies is around 20-30% CPU and 60-65% RAM.
Sometimes, however, one of the proxies has significantly less r (e.g. 5% CPU and 10% RAM). Looks like the proxies are not the bottleneck.
At the moment we have 2 linux proxies and 1 windows proxy in the pool - Does that make a difference? Should we go exclusively to Linux, for example? Is it better to use several slightly weaker linux proxies than fewer stronger ones, or does that not matter either?
Thank you!
-
- Enthusiast
- Posts: 81
- Liked: 9 times
- Joined: Sep 26, 2024 11:02 am
- Contact:
-
- Service Provider
- Posts: 94
- Liked: 16 times
- Joined: Jan 02, 2024 9:13 am
- Full Name: Pat
- Contact:
Re: Proxy pool balancing/ scalability question
I'm also interested in this, we have around 550 repos/550 jobs, and I have 8 windows proxy with 8vcpu/32gb RAM each in a proxy pool and I find that the CPU usage is usually 20-40% max and seems like the proxy threads count could be a lot higher on each one (it used to get close to 100% in v7 when we could tune the proxy load better) and the jobs now take a lot longer to run than they did before when each job was using a specific repo. I'm not sure if I should if it would be better to have more proxy's with less resources, or less proxys with more resources. So far the proxy pool seems slower and less efficient than in v7.
-
- Veeam Legend
- Posts: 368
- Liked: 114 times
- Joined: Nov 02, 2020 2:48 pm
- Full Name: Manuel Rios
- Location: Madrid, Spain
- Contact:
Re: Proxy pool balancing/ scalability question
In one post, they said that 1 Core = 4 Task item.
In our case we got 80 HT with a task limit of 320 items at time. And we're able to see 95-98% of CPU usage and 40GB of 64GB RAM used, and windows as OS.
We have the sense that linux proxy will be more sensitive to .net changes than Windows and you should block the patches. I think that actually Veeam priority system is a FIFO , doesnt care if backup task is full or increment. Just first-in-frist out.
In our case we got 80 HT with a task limit of 320 items at time. And we're able to see 95-98% of CPU usage and 40GB of 64GB RAM used, and windows as OS.
We have the sense that linux proxy will be more sensitive to .net changes than Windows and you should block the patches. I think that actually Veeam priority system is a FIFO , doesnt care if backup task is full or increment. Just first-in-frist out.
Service Provider | VMCE
Who is online
Users browsing this forum: sfey and 84 guests