Proxy Server maxed out on RAM usage

VMware specific discussions

Proxy Server maxed out on RAM usage

Veeam Logoby neilpotter » Tue May 09, 2017 8:56 am

We have recently moved from a "traditional" offsite backup to a Veeam Proxy server, to using an Offsite Copy Job.
This works soooo much better for us now and a lot quicker, bar one issue.
We have one backup of 12 "standard" VM's (DC's, SQL Server, Web Server etc) which runs like a dream, but we also have a separate backup for our main 6Tb File Server.
When this copy job sicks in, the proxy seems to take all the available RAM, and makes the server pretty much unusable for anything else. To the point that our monitoring software is telling us that server is offline for the first half an hour of that job.
So the VM is a 6Tb VMware 5.5 VM (4 Drives,3 of which are spanned to create a single volume).
The Proxy server (Veeam 9.5) has 12Gb RAM, and 8 cores to chew with.

Do I need to up the RAM, or is there something quicker and less costly that I can do to stop the server croaking when this backup kicks in?

Thanks
neilpotter
Influencer
 
Posts: 16
Liked: 1 time
Joined: Sat Sep 28, 2013 9:09 am
Full Name: Neil Potter

Re: Proxy Server maxed out on RAM usage

Veeam Logoby Vitaliy S. » Tue May 09, 2017 10:18 am

Proxy server usually consumes CPU resources, but not RAM. What's your bottleneck stats for this job? What is the weakest component, proxy or target repository? How many jobs do you run at the same time using this proxy server?
Vitaliy S.
Veeam Software
 
Posts: 19575
Liked: 1106 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Proxy Server maxed out on RAM usage

Veeam Logoby neilpotter » Tue May 09, 2017 10:45 am

Only ever one job at a time running.
The 12 x VM job runs up to 4 concurrent VM's and doesn't cause an issue.
The 1 x Large VM runs on it's own over a 100MB internet link over hardware VPN.

Stats from last few completed jobs are
Bottleneck tends to be network if there have been a lot of new files through the day, or Source if there are a lot of changes, I assume because of CBT?
Source bottle neck - Load Source 69% - Proxy 6% - Network 39% - Target 0%
Network bottleneck - Load Source 16% - Proxy 3% - Network 99% - Target 0%

For clarity, the proxy server is also the remote repo.

The RAM seems to maxed from when the job starts processing the info but I think it drops when files actually start to be transferred over. Its almost the first half an hour of the job when it's working out what needs to be transferred I think
neilpotter
Influencer
 
Posts: 16
Liked: 1 time
Joined: Sat Sep 28, 2013 9:09 am
Full Name: Neil Potter

Re: Proxy Server maxed out on RAM usage

Veeam Logoby Vitaliy S. » Tue May 09, 2017 11:04 am

neilpotter wrote:Bottleneck tends to be network if there have been a lot of new files through the day, or Source if there are a lot of changes, I assume because of CBT?

CBT is always used starting from the second run, so the stats should look more or less the same unless you're using different source datastore to retrieve VM data. In your case,if the more data you push the more it tends to be network bottleneck, then it could be indeed the expected behavior.

neilpotter wrote:For clarity, the proxy server is also the remote repo.

The RAM seems to maxed from when the job starts processing the info but I think it drops when files actually start to be transferred over. Its almost the first half an hour of the job when it's working out what needs to be transferred I think

Try increasing the RAM configuration, and let us know if that improves the server uptime or not.
Vitaliy S.
Veeam Software
 
Posts: 19575
Liked: 1106 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Proxy Server maxed out on RAM usage

Veeam Logoby neilpotter » Tue May 09, 2017 11:18 am

Ok, I will try that. Is there no way I can limit the amount of resource that Veeam can take instead?
neilpotter
Influencer
 
Posts: 16
Liked: 1 time
Joined: Sat Sep 28, 2013 9:09 am
Full Name: Neil Potter

Re: Proxy Server maxed out on RAM usage

Veeam Logoby Vitaliy S. » Tue May 09, 2017 12:01 pm

Limiting the resource usage will lower the processing rate. You might want to play with processes priority via Task Manager (just as a crazy idea), but you can limit concurrent tasks to 1 and check if you observe the same behavior or not.
Vitaliy S.
Veeam Software
 
Posts: 19575
Liked: 1106 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Proxy Server maxed out on RAM usage

Veeam Logoby neilpotter » Sun May 28, 2017 9:52 am 1 person likes this post

Quick update. I took the ram in both the Veeam server, and the remote Proxy / Repo from 12Tb to 32Tb, and it's revolutionised everything. It all runs a lot better, and much faster now.
No more offline warnings etc.
Thanks
neilpotter
Influencer
 
Posts: 16
Liked: 1 time
Joined: Sat Sep 28, 2013 9:09 am
Full Name: Neil Potter


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 18 guests