Calculate virtual memory needed to compact backup

Availability for the Always-On Enterprise

Calculate virtual memory needed to compact backup

Veeam Logoby Kas_Tigar » Fri Mar 10, 2017 2:30 pm

Hello,

we run into issue by compacting large full backup file (more than 20TB). Compact task failed with error like "Unable to allocate memory for storage metadata bank". The only one event in windows log at this time is the warning about virtual memory getting low. Virtual memory is controlled by the OS. So it should actually be auto increased. First I thought that OS increase virtual memory and only the space on drive C is the limitation of such increase.
But then I found the article with the hardcode limitation being 3 x RAM or 4 GB, whichever is larger
https://support.microsoft.com/en-us/hel ... of-windows

From this perspective I think If repository has 16GB RAM, virtual memory is increased during the compact job up to 48GB. If compact job is still not finished, OS is unable to increase the virtual memory and task is failed with the error "Unable to allocate memory for storage metadata bank".

The obvious workaround for this problem - assigning more RAM or manual configure virtual memory limits. But the question I have - how large should be virtual memory to compact 20TB of data? I wound experiment with size because such job takes days. So if someone has experience or know how I can calculate virtual memory needed for such task, would be great.
Thanks
Kas_Tigar
Influencer
 
Posts: 13
Liked: 2 times
Joined: Thu Jan 29, 2015 9:18 am

Re: Calculate virtual memmory needed to compact backup

Veeam Logoby JoshuaPostSAMC » Fri Mar 10, 2017 5:09 pm 1 person likes this post

I don't have a specific formula, but I had the same errors when doing a compact with multiple jobs with 2-3 TB files each. I went from 16 GB to 32 GB and all of those went away. Now that I'm doing Copy Jobs as well, I just increased to 48 GB and it still got into the 90% used range. More is better
JoshuaPostSAMC
Expert
 
Posts: 111
Liked: 15 times
Joined: Thu Jul 30, 2015 7:32 pm

Re: Calculate virtual memory needed to compact backup

Veeam Logoby foggy » Tue Mar 14, 2017 4:45 pm

Such memory consumption doesn't look expected, indeed. The process performing compact requires ~150 bytes per block, which even with the smallest block possible (256 KB) gives 12 GB of RAM for 20 TB backup file. What block size is used for this job? I also recommend opening a case with our technical support for a closer investigation. Thanks.
foggy
Veeam Software
 
Posts: 14482
Liked: 1057 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: brownbeargrrr, Google [Bot], nmdange, tkrajewski, VegaSE, yastt and 47 guests