Discussions specific to the Microsoft Hyper-V hypervisor
Post Reply
Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

On-host backup

Post by Robvil » Jan 05, 2017 10:33 am

Hi

I have 2 node 2012r2 cluster where my Veeam 9.5 is running as a VM. So all my backup are on-host.
Both servers has 64Gb memory - host1 has 30Gb free og host2 has 40Gb free.

Currently total of 4 VM´s, 2 windows and 2 linux, a total of 6TB data.

Each night during backup I get: "Pages/sec" (8229.0 Number) is above a defined threshold (1500.0 Number) from both members.
I know on-host backup demand ressources from the physical cluster servers, but I though 30-40Gb free memory would be enough. How demanding is on-host backup?

A off-host proxy does not even demand this much memory.

Robert

Shestakov
Veeam Software
Posts: 6974
Liked: 711 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: On-host backup

Post by Shestakov » Jan 05, 2017 5:49 pm

Hi Robert,
Does 8229 Pages/sec happen on the host or VM with VBR server? Do you have dynamic or static memory assignment?
My guess is that Backup Proxy VM (same as VBR server in your case) experiencing the lack of memory while the host still has a lot of it.
Thanks!

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 6:38 am

It happens on the physical hyper-V hosts and not inside the VM´s.
So it is not a VM memory issue, but a Hyper-V memory issue.

Mike Resseler
Product Manager
Posts: 5723
Liked: 606 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » Jan 06, 2017 6:46 am

Robert,

The number is really too high. Do you have the possibility to monitor "Memory\Available Mbytes" at the same time? If that also drops to 5% or less there is something eating memory and we need to figure out what that is.

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 6:57 am

Yes, i can try to run a scheduled Windows performance monitor job of memory/avaibable Mbytes.

I´m currently using Cacti to monitor the Hyper-V servers (using snmp and not WMI). The memory graph looks just fine - but snmp data is collected every 5 minuttes - so if the issue happens very shortly within 10-30 seconds, it will not show up on the graph. WIndows performance monitor will shows this if i choose a very low update interval.

Image

https://1drv.ms/i/s!AuOwdE3caya8hYwL25T1FXwf6jaPYQ

Mike Resseler
Product Manager
Posts: 5723
Liked: 606 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » Jan 06, 2017 7:11 am

Robert,

Yes, let's do that and see what the result is. From the memory graph there seems to be no issue. So it could indeed be very short (although that # would mean that in a short amount of time 30 GB of ram would be "eaten" somewhere...

- Did you have a fixed reserve of memory on your parent partition?
- What program is giving you that Pages/Sec is ... notification each night?
- And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?

Thanks
Mike

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 7:18 am

Did you have a fixed reserve of memory on your parent partition?
Answer: If you mean fixed meory allocation to my VM, then yes.

What program is giving you that Pages/Sec is ... notification each night?
Answer: Veeam One

And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?
Answer: What is the locale on your servers? Not sure what you mean?
My cacti server is on of the VM´s running on the Hyper-V

Mike Resseler
Product Manager
Posts: 5723
Liked: 606 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » Jan 06, 2017 7:27 am 1 person likes this post

No, you can actually reserve memory for your Hyper-V host. When you login to Hyper-V you login to the parent partition. Here is an interesting article about host reserve: http://www.aidanfinn.com/?p=19043

And with Locale I mean: Language and regional settings. I have seen sometimes that there are false alarms because of bad comma and point conversions. As in: 8,000 pages per sec becomes 8.000 pages per sec... (or vice versa...)

But let's certainly continue with that monitoring memory\Available Mbytes

Thanks

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 7:29 am

ahh okay, i´ll read the link. Thank.

My locate is english as language and regional is europe/Copenhagen.

Mike Resseler
Product Manager
Posts: 5723
Liked: 606 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » Jan 06, 2017 7:39 am

Also, thought of this: Can you run Hyper-V Custom Performance report? If I am not mistaken (I can't access my ONE environment at this moment) you can define there the memory usage for your host(s) so we see rather quickly what it has been doing. (Unless of course I am mistaken :-))

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 8:23 am

Here is a link to the pdf report.

https://1drv.ms/b/s!AuOwdE3caya8hYwMrGM5hbjBVLNQwg

What I do not understand is - the report only shows 1 time, where pages/sec was abnormally high on host1 - and every day, i get ONE mails saying pages/sec was to high.

Mike Resseler
Product Manager
Posts: 5723
Liked: 606 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » Jan 06, 2017 8:32 am

Looked at the report and you are right.

your average values are good. All under 500 which is considered Healthy.

I am lost now why you get this email every day. I think it might be time to log a support call so this can be investigated further.

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 9:20 am

I´ll do

Shestakov
Veeam Software
Posts: 6974
Liked: 711 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: On-host backup

Post by Shestakov » Jan 06, 2017 5:30 pm

Robert, since your custom performance report is only for Business hours:From 21:00 To 23:00 I`m curious what time do you get those alarms.
Once you have a support case number, please post it here for us to follow the investigation.

Mike, thanks for the Hyper-V host article.

Robvil
Expert
Posts: 129
Liked: 13 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil » Jan 06, 2017 8:43 pm

Case # 02031122

I took the report in time interval 21:00 to 23:00 as I only get the email alerts at 22:10 - 10 minutters after the backup of VM´s has startet (excluding the Veeam VM).
Only when backup is running. Never had the alarm during normal business hours.

Post Reply

Who is online

Users browsing this forum: No registered users and 18 guests