Host-based backup of VMware vSphere VMs.
Post Reply
FrenchBlue
Expert
Posts: 138
Liked: 23 times
Joined: Mar 18, 2021 6:04 pm
Contact:

Linux backup proxy scaling

Post by FrenchBlue »

Hello,

If there are no bottlenecks on the network or destination repositories and the cpu seems to be the limit, does a Linux backup proxy scale well with the number of cores of the processor(s)?
Is there an "optimal" number of cores where adding more will be more or less useless?
In general, is it better to add more cores, or rather increase the core clock frequency?

Thanks.
PetrM
Veeam Software
Posts: 3622
Liked: 608 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Linux backup proxy scaling

Post by PetrM »

Hello,

It is better to add more cores: first of all, we didn't test different specific clock frequencies, moreover increasing of cores number is usually better for applications that support multithreading. This article is quite interesting, it describes the pros and cons of each approach.

The number of cores that you need to have depends on the number of concurrent tasks that you're going to run. We don't recommend running more than 2 tasks per core. Also, I strongly recommend reading this help center page to get a better idea about concurrent tasks, it's relevant for both Windows and Linux proxies.

Thanks!
FrenchBlue
Expert
Posts: 138
Liked: 23 times
Joined: Mar 18, 2021 6:04 pm
Contact:

Re: Linux backup proxy scaling

Post by FrenchBlue »

Hello,

Thanks but this article is generic. It does not know how the Veeam processes on the proxy make use of the cpu, it depends on every application. Even applications that support multi-threading generally have some sort of sweet spot above which it is more or less useless to add more cores, it can also reach OS scheduling limits, have a common thread that will be the bottleneck etc...
From what I've read in the help center page though, can we guess that a new independent process is launched for each concurrent task running on the proxy?

Thanks.
PetrM
Veeam Software
Posts: 3622
Liked: 608 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Linux backup proxy scaling

Post by PetrM »

Hello,

Yes, a process per every concurrent task. Mainly, CPU resources are spent for the compression and data blocks hash compute operations. These operations are performed by an agent process that is started for every concurrent task. According to our tests in different environments, the best resource utilization can be achieved with 2 tasks per core.

Thanks!
FrenchBlue
Expert
Posts: 138
Liked: 23 times
Joined: Mar 18, 2021 6:04 pm
Contact:

Re: Linux backup proxy scaling

Post by FrenchBlue » 1 person likes this post

Thanks for clarification. One last question (for now 😁) , it it possible to have the proxy role on a hardened Linux repo, or should repo be the only role on the server in this case?
PetrM
Veeam Software
Posts: 3622
Liked: 608 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Linux backup proxy scaling

Post by PetrM » 1 person likes this post

Yes, it is possible but the server must be able to process the cumulative number of tasks specified for different roles, in your case repository and proxy. Also, keep in mind that for security reasons only Network mode is available for a proxy that has hardened repository role, more info here.

Thanks!
FrenchBlue
Expert
Posts: 138
Liked: 23 times
Joined: Mar 18, 2021 6:04 pm
Contact:

Re: Linux backup proxy scaling

Post by FrenchBlue »

Thanks, network mode is OK and we only use it, we cannot use hot add, too much VM stunning.
Post Reply

Who is online

Users browsing this forum: No registered users and 43 guests