Oversized VM's Report Too Agressive

Management reporting and documentation

Oversized VM's Report Too Agressive

Veeam Logoby deduplicat3d » Tue Nov 13, 2012 11:07 pm

When I launch the Oversized Virtual Machines by Memory report is shows the VM's Memory usage as very low and recommend I dramatically decrease memory. The numbers that are presented look more like active memory rather than consumed. Am I doing something wrong?
deduplicat3d
Enthusiast
 
Posts: 98
Liked: 11 times
Joined: Fri Nov 04, 2011 8:21 pm
Full Name: Corey

Re: Oversized VM's Report Too Agressive

Veeam Logoby deduplicat3d » Tue Nov 13, 2012 11:08 pm

PS in the Monitor tool when I click on a VM it shows active memory but in reporter when i run a report on the VM it shows consumed.
deduplicat3d
Enthusiast
 
Posts: 98
Liked: 11 times
Joined: Fri Nov 04, 2011 8:21 pm
Full Name: Corey

Re: Oversized VM's Report Too Agressive

Veeam Logoby Vitaliy S. » Wed Nov 14, 2012 7:51 am

Oversized Virtual Machines report is using "Memory Usage" counter which is calculated by dividing VM Memory Active by VM Memory Granted, we are not using memory consumed as this is a "false" metric to make sizing recommendations. For example, SQL Server VM will always report on 100% consumed memory, while only 20-30% might be actually actively used.

Be aware that while detecting oversized VMs, our engine searches for VMs with low average resource usage, detects the peak, and then adds 25% as a buffer to this peak usage, and finally makes the recommendations. Furthermore, if you want to verify the recommended configuration, you can drill-down to VM resource usage details and see what data our recommendation is based on.

Hope this helps!
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

[MERGED] Oversized VM report -- is it too aggressive?

Veeam Logoby cyberswb » Tue Dec 04, 2012 9:42 pm

I've run this report a couple of times and I'm always surprised at how aggressive the recommendations are for memory usage.

One client has a production Exchange 2007 VM servicing about 60 mailboxes. The existing VM is assigned 8 GB RAM but when I run the oversized VM report, it suggests cutting this down to 1.9 GB based on peak memory usage.

I guess part of me thinks would be way too low, despite what the active memory statistics would say -- you never see MS recommend running Windows Server 2008 x64 plus Exchange in a memory footprint that small. I've personally run Exchange 2010 (for 3 mailboxes..) in 2.5GB RAM VM but it was sloooowwww to manage, interactive logins were glacial, but the Exchange services seemed to run without issue.

Has anyone actually applied the recommendations as they come off the report without issue? Part of this is motivated by another client who "insists" that the 8 GB memory allocated to his SQL server is "too low" (he wants 32 to 64 GB) despite peak Memory Active being just under 2.25 GB.

Are there some cases where Memory Active isn't really accurate and the apparently inactive RAM is being used for local caching or something?
cyberswb
Influencer
 
Posts: 21
Liked: never
Joined: Mon Feb 01, 2010 7:41 pm
Full Name: Shawn Barnhart

Re: Oversized VM's Report Too Agressive

Veeam Logoby Vitaliy S. » Tue Dec 04, 2012 9:59 pm

Hi Shawn,

cyberswb wrote:I guess part of me thinks would be way too low, despite what the active memory statistics would say

What is the average and peak memory usage for this VM over 1 month? What is the current memory usage?

cyberswb wrote:Part of this is motivated by another client who "insists" that the 8 GB memory allocated to his SQL server is "too low" (he wants 32 to 64 GB) despite peak Memory Active being just under 2.25 GB.

What do other performance metrics (CPU Ready, Co-Stop, Disk Latency, the number of IOPs etc.) show for this VM? Do you have any performance issues with this SQL Server VM?

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Oversized VM's Report Too Agressive

Veeam Logoby cyberswb » Wed Dec 05, 2012 3:47 pm

Vitaliy S. wrote:
cyberswb wrote:I guess part of me thinks would be way too low, despite what the active memory statistics would say

What is the average and peak memory usage for this VM over 1 month? What is the current memory usage?


Current memory usage is about 13% now, peak over the past month was 23%.

What I find surprising is that if the "memory active" or "memory usage" statistics actually represent what the VM is really using actively for memory, then nearly all the conventional wisdom, advice, system requirements, and so on for memory are pretty much totally wrong. MS recommends 2 GB for Windows 2008 R2 and another 4 GB for Exchange 2010, plus 5 MB per mailbox. In most VM implementations I've seen active is never over 3 GB, even peak.

Anyway, that's why I'm question it because it's such a departure from conventional wisdom and vendor recommendations, which historically (for better or for worse) have been routinely considered ridiculous minimums and doubled in actual implementations.

Either memory usage/memory active is the real deal and we've been fooled all this time into using increasingly large amounts of RAM and assuming they're necessary, or those stats are an incomplete picture and there's some portion of the non-active/unused memory that are being used in ways we don't know.

Bottom line, I'm not saying it's wrong it's just such a disconnect from what I and everyone else has taken for granted that it seems like a paradigm changer.

Vitaliy S. wrote:
cyberswb wrote:Part of this is motivated by another client who "insists" that the 8 GB memory allocated to his SQL server is "too low" (he wants 32 to 64 GB) despite peak Memory Active being just under 2.25 GB.

What do other performance metrics (CPU Ready, Co-Stop, Disk Latency, the number of IOPs etc.) show for this VM? Do you have any performance issues with this SQL Server VM?


Generally performance is good and there are no complaints.

The SQL server is used for a very large application with a lot of data; on more than one occasion they've complained about performance that we generally attributed to a SAN system that was in need of an upgrade, but most of the time these problems are fixed with some kind of sheepish response about "bad queries" or "missing indexes" or some other problem unrelated to the server environment.

But I'm convinced that SQL guys are all impossible to please with performance data; they still want their own rack in the data center with physical servers they admin. Virtualization has been a loss of stature for the SQL guys.
cyberswb
Influencer
 
Posts: 21
Liked: never
Joined: Mon Feb 01, 2010 7:41 pm
Full Name: Shawn Barnhart

Re: Oversized VM's Report Too Agressive

Veeam Logoby Vitaliy S. » Thu Dec 06, 2012 12:52 pm

Hi Shawn,

cyberswb wrote: recommends 2 GB for Windows 2008 R2 and another 4 GB for Exchange 2010, plus 5 MB per mailbox. In most VM implementations I've seen active is never over 3 GB, even peak.

Yes, that's the trickiest part, especially when you try to size the Exchange/SQL Server VM. When you look at the active memory counter, it shows you the number of memory pages that have been recently accessed by the application, without taking into consideration pages that have been pre-allocated/reserved by the application, but not currently used. If you want to know the total amount of memory held by the VM, you need to review memory consumed metric. This metric is absolutely useless for SQL Servers/Exchange, as reading back through my response above these application will always consume all memory you've allocated to these VMs.

That said, in our oversized report we are using memory active (the peak usage) + 25% of buffer to give you the recommended amount of memory for your VMs. This should help you to avoid application performance problems after reducing the amount of memory allocated to the VM. Depending on the application and it's monthly workload you may want to add additional 10-15% of buffer to the recommended amount of RAM.

cyberswb wrote:Anyway, that's why I'm question it because it's such a departure from conventional wisdom and vendor recommendations, which historically (for better or for worse) have been routinely considered ridiculous minimums and doubled in actual implementations.

Either memory usage/memory active is the real deal and we've been fooled all this time into using increasingly large amounts of RAM and assuming they're necessary, or those stats are an incomplete picture and there's some portion of the non-active/unused memory that are being used in ways we don't know.

Bottom line, I'm not saying it's wrong it's just such a disconnect from what I and everyone else has taken for granted that it seems like a paradigm changer.


I fully agree with you, but I don't think that we were fooled. Installing additional memory on the physical server is not an easy task simply because you might not have available slots for additional RAM ;). That is why vendors might have recommended a bit more than it was usually required (in order to support future growth of the IT infrastructure).Today with the tools that are currently available, you can determine the amount of memory this is actually used by the application in your particular environment, so keeping vendors recommendations and your actual usage in mind, you can size your VMs in a proper way.

cyberswb wrote:Generally performance is good and there are no complaints.

The SQL server is used for a very large application with a lot of data; on more than one occasion they've complained about performance that we generally attributed to a SAN system that was in need of an upgrade, but most of the time these problems are fixed with some kind of sheepish response about "bad queries" or "missing indexes" or some other problem unrelated to the server environment.

But I'm convinced that SQL guys are all impossible to please with performance data; they still want their own rack in the data center with physical servers they admin. Virtualization has been a loss of stature for the SQL guys.

True :) Our report developers (SQL guys) think in the exactly same manner!
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Oversized VM's Report Too Agressive

Veeam Logoby Daveyd » Mon Dec 10, 2012 4:01 pm

Along these same lines, I just had an incident this morning where we migrated from an Oracle DB on a physical server to one on a VM. The Oracle DB was installed on a 2008 R2 VM and allocated 4GB of RAM and 2 vCPUs. This morning when users logged into the DB their application was slow to respond whereas it was not when the DB was on a physical server with the same allocated RAM. Looking at the RAM metrics for the past few hours, consumed was 4GB, and active was 0.9GB (Task Manager showed 3.8GB used with 300MB cached) yet the clients were painfully slow and timing out their server connections. The vendor recommended we up the RAM to 8GB. I did and the issue has gone away.

I showed the vendor that the active RAM was minimal and RAM constraint in the VM did not seem apparent but as a I stated, adding addition RAM to the VM improved client response time 100%.
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: Oversized VM's Report Too Agressive

Veeam Logoby Vitaliy S. » Mon Dec 10, 2012 4:03 pm

Ok, thanks David for your input! Do you have any ideas on how to better handle these situations (VMs with database servers inside)?
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Oversized VM's Report Too Agressive

Veeam Logoby Daveyd » Mon Dec 10, 2012 4:12 pm

I really wish I did especially when metrics are telling me everything is fine. Not blaming Veeam, of course. :)

Everything you read tells you do not rely at Task manager and rely on swap/ballooning/compression rates or reservations to determine if a VM is starved for memory and active memory/usage% should be a good indication of how well a VM is running in terms of RAM but in this case it was unreliable. The vendor stated he has seen this issue befire with Oracle DBs running in VMs and the issue has always been corrected by adding more RAM. Maybe it's something to do with the way Oracle utilizes RAM, I don't know. But it would have been nice to see a smoking gun and point it out.
Daveyd
Expert
 
Posts: 272
Liked: 10 times
Joined: Thu May 20, 2010 4:17 pm
Full Name: Dave DeLollis

Re: Oversized VM's Report Too Agressive

Veeam Logoby pufferdude » Wed Dec 12, 2012 11:08 pm

I too noticed these crazy-low memory recommendation and came here looking for answers and found this thread. I'm interested to know if anyone has actually followed the recommended memory allocations, because it just seems that they (at least some of them) can't possibly work correctly. For example, I have an admittedly low-use AD box at a remote site with 4GB memory and Veeam wants me to lower that to 0.2GB! Will Windows 2008R2 even BOOT with 256 *MB*? Another example... an old Win2003 VM running iMail with 750MB allocated and Veeam is recommending it be lowered to 100 *M* bytes.

I just don't understand how these recommendations can be remotely accurate or even doable.
pufferdude
Expert
 
Posts: 126
Liked: 4 times
Joined: Thu Jul 02, 2009 8:26 pm
Full Name: Jim

Re: Oversized VM's Report Too Agressive

Veeam Logoby Vitaliy S. » Thu Dec 13, 2012 10:53 am

Hi Jim,

The recommendations that are given in this report are based on the average active memory utilization for your VMs. You may want to add additional 25% of memory to the recommended size while reconfiguring the VMs.

pufferdude wrote: For example, I have an admittedly low-use AD box at a remote site with 4GB memory and Veeam wants me to lower that to 0.2GB!

Yes, I've seen the same behavior in my labs too... what if we put, let's say 1GBs, as a minimum recommendation in order not to show smth like 0.2 GBs? What do you think on that?

Thanks for the feedback!
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Oversized VM's Report Too Agressive

Veeam Logoby deduplicat3d » Fri Dec 14, 2012 3:14 am 1 person likes this post

I don't think that you should put a minimum into Veeam. I like having Veeam give the recommendations that it feels are true and then the Sys Admins can use best judgement. I have some linux servers that I've decreased to those levels because of Veeam, I was skeptical at first, but now I'm a believer!
deduplicat3d
Enthusiast
 
Posts: 98
Liked: 11 times
Joined: Fri Nov 04, 2011 8:21 pm
Full Name: Corey

[MERGED] This can't be right?

Veeam Logoby chris352 » Sun Dec 23, 2012 2:52 am

Can anyone shed some light on this? Each of our hosts has 64GB or ram and we're serving some 200+ users... Exchange 2010, SBS 2011, over a dozen Xenapp hosts etc and Veeam report is telling me to drop ram???
I thought we were barely scraping by with what we had...


Image
chris352
Novice
 
Posts: 9
Liked: never
Joined: Fri Oct 14, 2011 10:34 pm
Full Name: Chris Oleson

Re: This can't be right?

Veeam Logoby Vitaliy S. » Sun Dec 23, 2012 5:27 pm

Hi Chris,

This report is based on active memory usage counter, which shows how much memory your VMs are actively using. As far as I can see your VMs are not using all memory you've allocated to them, that is why you have that amount of RAM in the recommendations section.

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19563
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Next

Return to Reporting



Who is online

Users browsing this forum: No registered users and 2 guests