-
- Novice
- Posts: 4
- Liked: never
- Joined: Jan 08, 2019 7:54 pm
- Full Name: R. Bello
- Contact:
Max Memory Usage
Hello all, semi-new to Veeam and need a little bit of help.
I'm running Veeam B&R 9.5 U3 (9.5.0.1922) on a standalone Dell R620 Windows 2012 R2 Standard server (not on our domain). The server has 96GB of physical memory (this server was re-tasked so it had that much memory), 220GB C: drive (4 - 111.25 GB SSD's - RAID 10) and 8.5TB D: drive for Veeam Repository. The D: drive is connected via iSCSI to an EqualLogic PS4100.
I currently have only 1 backup job connected to our vCenter (6.7 U1, I've made the registry fix change) which has 3 Dell R630 hosts (also 6.7 U1) and I'm trying to back up 8 guest VM's. The job starts fine, the processing rate was always above 80 MB/s and the bottleneck was the source. As soon as the job starts, memory used starts rising until it maxes out and the job freezes along with the server; I lose connectivity to the server but I'm able to RDP back in to kill the Veeam process. After doing that, the memory usage stays the same; only a reboot of the server resets the memory in use.
I downloaded SQL Management Studio and made changes for SQL to only use 32GB of memory, I've downloaded Cacheset and changed the Working set maximum to 32GB but the backup job still ends up using all the memory and eventually fails. During the job, RAMMap shows Nonpaged Pool and Mapped File using most of the memory. Not sure where else to check to troubleshoot this issue
I've read other posts regarding memory usage but none of their resolutions seem to work for me.
Any help is appreciated and if I left out any specs that may be helpful for troubleshooting, let me know and I'd be happy to provide them.
Thanks in advance for any help.
-RB
I'm running Veeam B&R 9.5 U3 (9.5.0.1922) on a standalone Dell R620 Windows 2012 R2 Standard server (not on our domain). The server has 96GB of physical memory (this server was re-tasked so it had that much memory), 220GB C: drive (4 - 111.25 GB SSD's - RAID 10) and 8.5TB D: drive for Veeam Repository. The D: drive is connected via iSCSI to an EqualLogic PS4100.
I currently have only 1 backup job connected to our vCenter (6.7 U1, I've made the registry fix change) which has 3 Dell R630 hosts (also 6.7 U1) and I'm trying to back up 8 guest VM's. The job starts fine, the processing rate was always above 80 MB/s and the bottleneck was the source. As soon as the job starts, memory used starts rising until it maxes out and the job freezes along with the server; I lose connectivity to the server but I'm able to RDP back in to kill the Veeam process. After doing that, the memory usage stays the same; only a reboot of the server resets the memory in use.
I downloaded SQL Management Studio and made changes for SQL to only use 32GB of memory, I've downloaded Cacheset and changed the Working set maximum to 32GB but the backup job still ends up using all the memory and eventually fails. During the job, RAMMap shows Nonpaged Pool and Mapped File using most of the memory. Not sure where else to check to troubleshoot this issue
I've read other posts regarding memory usage but none of their resolutions seem to work for me.
Any help is appreciated and if I left out any specs that may be helpful for troubleshooting, let me know and I'd be happy to provide them.
Thanks in advance for any help.
-RB
-
- Product Manager
- Posts: 14724
- Liked: 1705 times
- Joined: Feb 04, 2013 2:07 pm
- Full Name: Dmitry Popov
- Location: Prague
- Contact:
Re: Max Memory Usage
Hello and welcome to the community rbcx.
Thank you for the detailed explanation of your problem. The described behavior does not look like expected, so I please raise a support case and let our team investigate the debug logs. Do not forget to share the case ID in this thread for future reference. Cheers!
Thank you for the detailed explanation of your problem. The described behavior does not look like expected, so I please raise a support case and let our team investigate the debug logs. Do not forget to share the case ID in this thread for future reference. Cheers!
-
- VP, Product Management
- Posts: 7080
- Liked: 1511 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: Max Memory Usage
Beside the support case.
Sometimes it happens that the Antivirus solution tries to scan our backup files (and high change rate) or even try to scan the network traffic. This leaded to high memory and cpu utilization.
Can you please check in Windows Task Manager which task will consum all the memory. Then we can maybe guess which component it is and why.
Please check as well the answer marked here:
http://www.tomshardware.co.uk/answers/i ... emory.html
It describe what happened and in that case it was hardware driver related.
Sometimes it happens that the Antivirus solution tries to scan our backup files (and high change rate) or even try to scan the network traffic. This leaded to high memory and cpu utilization.
Can you please check in Windows Task Manager which task will consum all the memory. Then we can maybe guess which component it is and why.
Please check as well the answer marked here:
http://www.tomshardware.co.uk/answers/i ... emory.html
It describe what happened and in that case it was hardware driver related.
-
- Novice
- Posts: 4
- Liked: never
- Joined: Jan 08, 2019 7:54 pm
- Full Name: R. Bello
- Contact:
Re: Max Memory Usage
Case #03354221 was created.
No AV has been installed on this server...yet, so no scanning being done.
I will run the Windows Memory Diagnostic tool and will also look into the Windows Driver Kit if needed.
I'll update this post if anything is found and/or when support responds.
Thanks.
No AV has been installed on this server...yet, so no scanning being done.
I will run the Windows Memory Diagnostic tool and will also look into the Windows Driver Kit if needed.
I'll update this post if anything is found and/or when support responds.
Thanks.
-
- Novice
- Posts: 4
- Liked: never
- Joined: Jan 08, 2019 7:54 pm
- Full Name: R. Bello
- Contact:
Re: Max Memory Usage
Windows Memory Diagnostic did not find any issues.
I updated to 9.5 U4 as instructed by support and the job still had the same results; memory maxed, job failed and server inaccessible. As I was copying the logs from the Veeam server to my computer, I noticed it took over a minute to copy a 2MB file. I started looking at the network connections for the Veeam server and saw that there was a network bridge for the 2 LAN NICs. I disabled the bridge and configured NIC teaming instead.
I ran the job and it finished successfully!
I updated to 9.5 U4 as instructed by support and the job still had the same results; memory maxed, job failed and server inaccessible. As I was copying the logs from the Veeam server to my computer, I noticed it took over a minute to copy a 2MB file. I started looking at the network connections for the Veeam server and saw that there was a network bridge for the 2 LAN NICs. I disabled the bridge and configured NIC teaming instead.
I ran the job and it finished successfully!
-
- VP, Product Management
- Posts: 7080
- Liked: 1511 times
- Joined: May 04, 2011 8:36 am
- Full Name: Andreas Neufert
- Location: Germany
- Contact:
Re: Max Memory Usage
Wow... LAN bridging ... would never expected this...
Thanks for sharing
Thanks for sharing
-
- Novice
- Posts: 4
- Liked: never
- Joined: Jan 08, 2019 7:54 pm
- Full Name: R. Bello
- Contact:
Re: Max Memory Usage
I'd also like to add that when the job ran successfully, memory usage was never above 6%.
Who is online
Users browsing this forum: No registered users and 76 guests