Veeam B&R/Proxy/WAN Accelator resource hog?

[damadhatter]

I use Veeam B&R to provide onsite backups & offsite copies to various sites to a central repository in my data center. My sites each have a single VMWare ESXi host w/ 2 sockets & 32GBs of RAM, most having 1-2 VMs per host. The onsite job runs to a target on a Windows VM residing on the same host, it's repository is a WIndows iSCSI target to a NAS. Each site is connected to my main site via a VPN tunnel so no need for the Cloud Provider portion of Veeam to do my offsite copy jobs (it just costs more $$$$ to be spent). So far I have about 8 sites and I am looking to double that number this year.

Since implementing Veeam over a year ago it has been the most unreliable product. I have a failure almost daily, and it seems to jump around from site to site. I have fought support up & down with the various issues to really try and narrow down the problem. My issues have always been more so with the offsite copy jobs. I have been told to add more memory & bandwidth to my offsite destination and the offsite copy jobs seem to be more stable with the occasional failure now only occurring every 3-4 days.

Since stabilizing things with the offsite copy jobs I have recently run into an issue with the onsite jobs now failing frequently. These failures were first citing a memory issue and support was able to find that this WASNT a resource issue but an actual bug with the WAN Accelerators (I am on day #2 of the fixed WAN Accelerator exe). Now that issue is resolved and I have a new one citing SQL unavailable (this didn't happen in v8). Support is telling me I all of a sudden need more resources on my various source site Veeam VMs. I typically run 2 cores & 6-8GBs of RAM for each sites Veeam VM running Veeam B&R, Proxy & WAN Accelerator.

I was given this by support

•B&R: 1 core; 4GB + 500mb/task
•Proxy: 2CPU min; 2GB +200mb/task
•Repo: 4 GB RAM, plus up to 2 GB RAM (32-bit OS) or up to 4 GB RAM (64-bit OS) for each concurrent job depending on backup chain’s length and backup files sizes.
•WAN: 1 core; 8GB RAM

Technically speaking, this server as an AIO (all-in-one): Veeam Server + SQL (for Veeam) + Proxy + Repo + WAN should = over 20GB of RAM

These specs are a bit insane to be backing up 1-2 VMs!

What are others doing for a similar setup? I can't imagine Veeam requiring double the resources than my sites file/application server would.

Any feedback would be greatly appreciated!!!

[chrisdearden]

With so few machines per site ( and assuming they are stable, ie not adding in new ones very often ) - have you seen what the impact of not using WAN acceleration is ?

[damadhatter]

The job time almost doubles :/ I only run the job once per day, it is typically only 1-3GBs of changes a day per site.

Each site is very static, usually a DC & a AIO file/app/print server

[chrisdearden]

for the copy job ?
but does it still complete within an acceptable window ?

[alanbolte]

New caching behavior was introduced to the repositories in v8 update 2. You can reduce repository RAM usage by disabling this caching behavior, but it's hard to say how much that will slow down your backups. Support should be able to tell you the registry value. If you disable inline deduplication in the job settings, this cache will be disabled without needing the registry value, and you will likely see further reductions in memory usage, at the cost of larger backup files. I don't recall whether you need to run an active full for that setting to take effect.

Keeping your event history retention short can reduce the memory usage of the backup server; while that primarily affects the GUI, it can help somewhat overall.

[damadhatter]

for the copy job ?
but does it still complete within an acceptable window ?

I like the speed, and will require it if I plan to run this job more frequently.

[damadhatter]

Keeping your event history retention short can reduce the memory usage of the backup server; while that primarily affects the GUI, it can help somewhat overall.

Where could I make this change?

Do my specs seem a little under what they should be for what I am trying to accomplish?

[DaveWatkins]

I can't lend any help, but this caught my eye

•Repo: 4 GB RAM, plus up to 2 GB RAM (32-bit OS)

How exactly does a 32bit OS address more than 4GB of RAM?! I'm assuming their recommendation was +2GB RAM per task too, so with multiple tasks they are saying you should allocate 8 or 10 or 12GB of RAM to a 32bit OS?

I agree these recommendations seem hugely excessive

[damadhatter]

Well I looked at the total of 20GBs and skipped the details above haha, who runs a 32bit OS?!

[alanbolte]

Where could I make this change?

https://helpcenter.veeam.com/backup/vsp ... tions.html
or, if you are using Enterprise Manager:
https://helpcenter.veeam.com/backup/em/ ... tings.html

I should also clarify that this is primarily relevant to resource usage by the configuration database, moreso than the backup service or console. If you're using the default SQL Express instance, its buffer pool is limited to 1GB regardless.

You'll have to wait a while for the change to take effect.

[damadhatter]

Thanks for this info. I am using SQL express across the board. Are you speaking about my actual Veeam B&R machines, in regards to the 1GB RAM usage?

[alanbolte]

The configuration database is located on the Veeam B&R machine by default. It can be located elsewhere.

[damadhatter]

I have mine installed default so those would be locally. I wasn't aware it was limited to 1GB of RAM usage. Do you have some documentation on that? I would like to share with the tech I am working with.

[Gostev]

•B&R: 1 core; 4GB + 500mb/task
•Proxy: 2CPU min; 2GB +200mb/task
•Repo: 4 GB RAM, plus up to 2 GB RAM (32-bit OS) or up to 4 GB RAM (64-bit OS) for each concurrent job depending on backup chain’s length and backup files sizes.
•WAN: 1 core; 8GB RAM

For all-in-one installations, feel free to substract 2GB from each role, as those 2GB are allotted to the OS itself in our requirements assuming each component is installed on the dedicated server. Thanks!

How exactly does a 32bit OS address more than 4GB of RAM?!

Using PAE (Physical Address Extension).

I agree these recommendations seem hugely excessive

For 1-2 VMs, sure. But this does not exactly represent a typical Veeam deployment. We had these recommendations hugely conservative before, and had too many support cases caused by lack of memory in larger deployments, to the point when support folks came to us and made us bump these up.