Host-based backup of Microsoft Hyper-V VMs.
Post Reply
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

On-host backup

Post by Robvil »

Hi

I have 2 node 2012r2 cluster where my Veeam 9.5 is running as a VM. So all my backup are on-host.
Both servers has 64Gb memory - host1 has 30Gb free og host2 has 40Gb free.

Currently total of 4 VM´s, 2 windows and 2 linux, a total of 6TB data.

Each night during backup I get: "Pages/sec" (8229.0 Number) is above a defined threshold (1500.0 Number) from both members.
I know on-host backup demand ressources from the physical cluster servers, but I though 30-40Gb free memory would be enough. How demanding is on-host backup?

A off-host proxy does not even demand this much memory.

Robert
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: On-host backup

Post by Shestakov »

Hi Robert,
Does 8229 Pages/sec happen on the host or VM with VBR server? Do you have dynamic or static memory assignment?
My guess is that Backup Proxy VM (same as VBR server in your case) experiencing the lack of memory while the host still has a lot of it.
Thanks!
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

It happens on the physical hyper-V hosts and not inside the VM´s.
So it is not a VM memory issue, but a Hyper-V memory issue.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler »

Robert,

The number is really too high. Do you have the possibility to monitor "Memory\Available Mbytes" at the same time? If that also drops to 5% or less there is something eating memory and we need to figure out what that is.
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

Yes, i can try to run a scheduled Windows performance monitor job of memory/avaibable Mbytes.

I´m currently using Cacti to monitor the Hyper-V servers (using snmp and not WMI). The memory graph looks just fine - but snmp data is collected every 5 minuttes - so if the issue happens very shortly within 10-30 seconds, it will not show up on the graph. WIndows performance monitor will shows this if i choose a very low update interval.

Image

https://1drv.ms/i/s!AuOwdE3caya8hYwL25T1FXwf6jaPYQ
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler »

Robert,

Yes, let's do that and see what the result is. From the memory graph there seems to be no issue. So it could indeed be very short (although that # would mean that in a short amount of time 30 GB of ram would be "eaten" somewhere...

- Did you have a fixed reserve of memory on your parent partition?
- What program is giving you that Pages/Sec is ... notification each night?
- And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?

Thanks
Mike
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

Did you have a fixed reserve of memory on your parent partition?
Answer: If you mean fixed meory allocation to my VM, then yes.

What program is giving you that Pages/Sec is ... notification each night?
Answer: Veeam One

And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?
Answer: What is the locale on your servers? Not sure what you mean?
My cacti server is on of the VM´s running on the Hyper-V
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler » 1 person likes this post

No, you can actually reserve memory for your Hyper-V host. When you login to Hyper-V you login to the parent partition. Here is an interesting article about host reserve: http://www.aidanfinn.com/?p=19043

And with Locale I mean: Language and regional settings. I have seen sometimes that there are false alarms because of bad comma and point conversions. As in: 8,000 pages per sec becomes 8.000 pages per sec... (or vice versa...)

But let's certainly continue with that monitoring memory\Available Mbytes

Thanks
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

ahh okay, i´ll read the link. Thank.

My locate is english as language and regional is europe/Copenhagen.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler »

Also, thought of this: Can you run Hyper-V Custom Performance report? If I am not mistaken (I can't access my ONE environment at this moment) you can define there the memory usage for your host(s) so we see rather quickly what it has been doing. (Unless of course I am mistaken :-))
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

Here is a link to the pdf report.

https://1drv.ms/b/s!AuOwdE3caya8hYwMrGM5hbjBVLNQwg

What I do not understand is - the report only shows 1 time, where pages/sec was abnormally high on host1 - and every day, i get ONE mails saying pages/sec was to high.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: On-host backup

Post by Mike Resseler »

Looked at the report and you are right.

your average values are good. All under 500 which is considered Healthy.

I am lost now why you get this email every day. I think it might be time to log a support call so this can be investigated further.
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

I´ll do
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: On-host backup

Post by Shestakov »

Robert, since your custom performance report is only for Business hours:From 21:00 To 23:00 I`m curious what time do you get those alarms.
Once you have a support case number, please post it here for us to follow the investigation.

Mike, thanks for the Hyper-V host article.
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: On-host backup

Post by Robvil »

Case # 02031122

I took the report in time interval 21:00 to 23:00 as I only get the email alerts at 22:10 - 10 minutters after the backup of VM´s has startet (excluding the Veeam VM).
Only when backup is running. Never had the alarm during normal business hours.
Post Reply

Who is online

Users browsing this forum: No registered users and 24 guests