On-host backup

[Robvil]

Hi

I have 2 node 2012r2 cluster where my Veeam 9.5 is running as a VM. So all my backup are on-host.
Both servers has 64Gb memory - host1 has 30Gb free og host2 has 40Gb free.

Currently total of 4 VM´s, 2 windows and 2 linux, a total of 6TB data.

Each night during backup I get: "Pages/sec" (8229.0 Number) is above a defined threshold (1500.0 Number) from both members.
I know on-host backup demand ressources from the physical cluster servers, but I though 30-40Gb free memory would be enough. How demanding is on-host backup?

A off-host proxy does not even demand this much memory.

Robert

[Shestakov]

Hi Robert,
Does 8229 Pages/sec happen on the host or VM with VBR server? Do you have dynamic or static memory assignment?
My guess is that Backup Proxy VM (same as VBR server in your case) experiencing the lack of memory while the host still has a lot of it.
Thanks!

[Robvil]

It happens on the physical hyper-V hosts and not inside the VM´s.
So it is not a VM memory issue, but a Hyper-V memory issue.

[Mike Resseler]

Robert,

The number is really too high. Do you have the possibility to monitor "Memory\Available Mbytes" at the same time? If that also drops to 5% or less there is something eating memory and we need to figure out what that is.

[Robvil]

Yes, i can try to run a scheduled Windows performance monitor job of memory/avaibable Mbytes.

I´m currently using Cacti to monitor the Hyper-V servers (using snmp and not WMI). The memory graph looks just fine - but snmp data is collected every 5 minuttes - so if the issue happens very shortly within 10-30 seconds, it will not show up on the graph. WIndows performance monitor will shows this if i choose a very low update interval.



https://1drv.ms/i/s!AuOwdE3caya8hYwL25T1FXwf6jaPYQ

[Mike Resseler]

Robert,

Yes, let's do that and see what the result is. From the memory graph there seems to be no issue. So it could indeed be very short (although that # would mean that in a short amount of time 30 GB of ram would be "eaten" somewhere...

- Did you have a fixed reserve of memory on your parent partition?
- What program is giving you that Pages/Sec is ... notification each night?
- And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?

Thanks
Mike

[Robvil]

Did you have a fixed reserve of memory on your parent partition?
Answer: If you mean fixed meory allocation to my VM, then yes.

What program is giving you that Pages/Sec is ... notification each night?
Answer: Veeam One

And really stupid question... What is the locale on your servers? And on the server monitoring your Hyper-V servers?
Answer: What is the locale on your servers? Not sure what you mean?
My cacti server is on of the VM´s running on the Hyper-V

[Mike Resseler]

No, you can actually reserve memory for your Hyper-V host. When you login to Hyper-V you login to the parent partition. Here is an interesting article about host reserve: http://www.aidanfinn.com/?p=19043

And with Locale I mean: Language and regional settings. I have seen sometimes that there are false alarms because of bad comma and point conversions. As in: 8,000 pages per sec becomes 8.000 pages per sec... (or vice versa...)

But let's certainly continue with that monitoring memory\Available Mbytes

Thanks

[Robvil]

ahh okay, i´ll read the link. Thank.

My locate is english as language and regional is europe/Copenhagen.

[Mike Resseler]

Also, thought of this: Can you run Hyper-V Custom Performance report? If I am not mistaken (I can't access my ONE environment at this moment) you can define there the memory usage for your host(s) so we see rather quickly what it has been doing. (Unless of course I am mistaken )

[Robvil]

Here is a link to the pdf report.

https://1drv.ms/b/s!AuOwdE3caya8hYwMrGM5hbjBVLNQwg

What I do not understand is - the report only shows 1 time, where pages/sec was abnormally high on host1 - and every day, i get ONE mails saying pages/sec was to high.

[Mike Resseler]

Looked at the report and you are right.

your average values are good. All under 500 which is considered Healthy.

I am lost now why you get this email every day. I think it might be time to log a support call so this can be investigated further.

[Robvil]

I´ll do

[Shestakov]

Robert, since your custom performance report is only for Business hours:From 21:00 To 23:00 I`m curious what time do you get those alarms.
Once you have a support case number, please post it here for us to follow the investigation.

Mike, thanks for the Hyper-V host article.

[Robvil]

Case # 02031122

I took the report in time interval 21:00 to 23:00 as I only get the email alerts at 22:10 - 10 minutters after the backup of VM´s has startet (excluding the Veeam VM).
Only when backup is running. Never had the alarm during normal business hours.