High Memory Usage On Restore/Replication

Hyper-V specific discussions

High Memory Usage On Restore/Replication

Veeam Logoby mbroaders » Thu Oct 10, 2013 6:56 pm

Case ID 00454092
Server 2012 hyper-v cluster environment. Production environment on Cisco UCS C240 servers with EMC VNXe Storage. DR Environment on Dell R710 servers with a Dell Powervault MD3200i.
Issue occurs in Veeam 6.5 and Veeam 7 (with update 1).
Server 2012 has all hyper-v and failover clustering hotfixes installed linked from http://blogs.technet.com/b/askcore/arch ... fixes.aspx .
Veeam 7 is installed on a 2012 VM on both the production environment and in the DR environment.
Issue is when restoring a virtual machine the memory usage goes up by approximately 20GB which causes the production hosts to max out and make vms unresponsive. Can reproduce 100% of the time, issue occurs on both production and dr servers although as the DR servers are not as loaded memory doesnt go to 100% but still see up to 20GB increase in RAM usage. When performing a backup there is no noticeable problem on the hosts. When permorming a replication from production to DR i see memory usage on the DR side go up again.

Support have told me it is a known issue with 2012. I can see known issues with backups of CSVs but not restores. Would anyone be able to confirm what bug this is and if it is reproducible on their systems?
mbroaders
Service Provider
 
Posts: 17
Liked: 1 time
Joined: Tue May 15, 2012 9:06 am
Full Name: Martin Broaders

Re: High Memory Usage On Restore/Replication

Veeam Logoby nefes » Fri Oct 11, 2013 5:47 am

Martin, do you see what consumes that memory?

As far as I know, Win2012 use it's memory to hold files it copies in the system cache, so when it comes to copying large VHDs or VHDXs, they are stored in host's RAM if it has some spare space, leading to behavior you see.

You can try to reproduce it by copying some big file with Windows itself and see what happens to your host.
nefes
Veeam Software
 
Posts: 534
Liked: 125 times
Joined: Mon Dec 10, 2012 8:44 am
Full Name: Nikita Efes

Re: High Memory Usage On Restore/Replication

Veeam Logoby mbroaders » Tue Oct 15, 2013 9:26 am

Thanks for the reply. I followed the below procedure to test :

1. Copied a 60GB backup file from the backup location (an iomega nas) to the local file system of a 2012 host. Memory increased by just 2% during the restore
2. Repeated this test but used a CSV for the destination. Same results. 2% increase in memory.
3. Ran a full vm restore to the same host and same csv as 2. Memory increased by 45%.

Have been asked for log files which i am uploading at present.
mbroaders
Service Provider
 
Posts: 17
Liked: 1 time
Joined: Tue May 15, 2012 9:06 am
Full Name: Martin Broaders

Re: High Memory Usage On Restore/Replication

Veeam Logoby nshoobridge » Wed Oct 16, 2013 4:15 pm

Hi,

i have exactly the same issuse. My host is using all memory when making backup. I also have the issue when I copie big files. So I think it is related to the caching mechanism of Mircrosoft. Is there a way to disable that function?
The only workaround when copying files I have to use xcopy. XCOPY with option /J (Copies files without buffering) the system uses no memory and copies the files like a charm.

But what to do when making backupjobs?

Cheers,
Neils
nshoobridge
Lurker
 
Posts: 2
Liked: never
Joined: Fri Aug 02, 2013 4:28 am
Full Name: Neils

Re: High Memory Usage On Restore/Replication

Veeam Logoby foggy » Thu Oct 17, 2013 2:12 pm

First of all, it should be verified, what processes consume memory during Veeam B&R activity (using Task Manager, for example).

nshoobridge wrote:So I think it is related to the caching mechanism of Mircrosoft. Is there a way to disable that function?

HOW TO: Manually Turn Disk Write Caching On or Off

Though the instruction is for W2008, it still applies to W2012. Note, that altering this setting is not recommended.
foggy
Veeam Software
 
Posts: 14753
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: High Memory Usage On Restore/Replication

Veeam Logoby nshoobridge » Thu Oct 17, 2013 7:56 pm

Hi,

thanks for your mail. I missed to say that source an destination are iSCSI devices and not local drive. So I'm not able to disable write cache in that way.
/C
nshoobridge
Lurker
 
Posts: 2
Liked: never
Joined: Fri Aug 02, 2013 4:28 am
Full Name: Neils

Re: High Memory Usage On Restore/Replication

Veeam Logoby doobie » Thu Jul 10, 2014 8:08 am

Did you ever find a resolution to this issue. I have the same issue and have had a ticket outstanding with Veeam for over a month. It only does it when Veeam preallocates the space for the vhd. Doesn't happen when large files are copied.

Only difference is I'm running with direct attached storage rather than ISCSI, but need to get this sorted otherwise Veeam is no good for me.
doobie
Lurker
 
Posts: 1
Liked: never
Joined: Thu Jul 10, 2014 8:06 am
Full Name: Darren Stokes

Re: High Memory Usage On Restore/Replication

Veeam Logoby Vitaliy S. » Thu Jul 10, 2014 4:23 pm

Darren, can you please post your case ID? Also can you please clarify with the Task Manager what process utilizes most memory on the host?
Vitaliy S.
Veeam Software
 
Posts: 19573
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: High Memory Usage On Restore/Replication

Veeam Logoby nublet » Fri Mar 27, 2015 5:46 am

Has anyone able to fix this problem on restore high mem usage on server 2008 r2? Thanks.
nublet
Novice
 
Posts: 5
Liked: never
Joined: Wed Jul 03, 2013 1:59 am
Full Name: Nublet

Re: High Memory Usage On Restore/Replication

Veeam Logoby foggy » Fri Mar 27, 2015 3:29 pm

What Veeam B&R version are you using and what processes take the most memory on the host? Have you already contacted technical support?
foggy
Veeam Software
 
Posts: 14753
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: High Memory Usage On Restore/Replication

Veeam Logoby nublet » Mon Mar 30, 2015 12:02 am

We are using Veeam 8.0
Yeah I have contacted technical support, they mention its a caching problem on the server, but not able to fix it.
Do you know if there's anything about this?
nublet
Novice
 
Posts: 5
Liked: never
Joined: Wed Jul 03, 2013 1:59 am
Full Name: Nublet

Re: High Memory Usage On Restore/Replication

Veeam Logoby foggy » Mon Mar 30, 2015 9:59 am

Could you please provide your support case ID?
foggy
Veeam Software
 
Posts: 14753
Liked: 1083 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

[MERGED]: High Cache Memory Veeam HyperV

Veeam Logoby iroal » Mon Apr 25, 2016 8:37 am

Hi.

I'm testing Veeam Backup for HyperV.

I've a problem with the Cached Memory, It grows to much and collapse my Server.

is there any option to limit the cache in Veeam Backup?

I tried the AgentReadOnlyCache set to 0, but the problem continue.

Thanks.

Image
iroal
Lurker
 
Posts: 1
Liked: never
Joined: Mon Apr 25, 2016 8:31 am

Re: High Memory Usage On Restore/Replication

Veeam Logoby PTide » Mon Apr 25, 2016 4:41 pm

Hi,

Your post has been merged into an existing topic. Please review it and make sure that it's VBR what causes high memory consumption. If that is the case then please contact support team and post your case ID here.

Thank you.
PTide
Veeam Software
 
Posts: 3022
Liked: 247 times
Joined: Tue May 19, 2015 1:46 pm

Re: High Memory Usage On Restore/Replication

Veeam Logoby stephc_msft » Wed May 04, 2016 12:54 pm

Please confirm if you are running on Server 2012 or 2012R2 as the hyper-V host
as 2012R2 should not be having the issues to the same extent
stephc_msft
Lurker
 
Posts: 1
Liked: never
Joined: Wed May 04, 2016 12:35 pm
Full Name: Stephen Cole


Return to Microsoft Hyper-V



Who is online

Users browsing this forum: DGrinev and 18 guests