V7 - VMware Issue - False alerts

Monitoring and reporting for Veeam Backup & Replication, VMware vSphere and Microsoft Hyper-V

Re: V7 - False alerts

Veeam Logoby Vitaliy S. » Tue Sep 03, 2013 6:21 pm

Hmm...at what times do you usually get these spikes? Are there any sort of maintenance tasks scheduled at the time when spikes occur?
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: V7 - False alerts

Veeam Logoby TomBlue » Tue Sep 03, 2013 6:36 pm

Spikes are occuring unregular. I cannot find any relation to jobs running on this or other servers. Here's a screenshot of todays metrics: http://sdrv.ms/18p5TZR. Because this is 1 day period values are a little "foggy" - you do not see spikes that are only half of others. I counted the errors of last 24 hours: There are approx. 25 and no recognisable pattern.

[Update]: There have been some occurences last minutes with maximum values of 127131031 ms , that's 35 hours. That could not be any real latency. At the same time I had very low DAVG/cms values esxtop.

Anyway, thanks for your help, I will try to get any VMware support - or simply ignore this issue, because there is no constraint in "real life" :?
TomBlue
Influencer
 
Posts: 14
Liked: 3 times
Joined: Fri Mar 23, 2012 8:19 am
Full Name: Thomas Blühmann

Re: V7 - False alerts

Veeam Logoby foggy » Tue Sep 03, 2013 10:04 pm

I'm trying to reproduce this issue in my labs, but I haven't been successful so far. If VMware support nails down this issue, I would appreciate if could come back and update this topic with their findings.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: V7 - False alerts

Veeam Logoby GL@MO » Wed Sep 04, 2013 6:24 pm

Experiencing the same "problem"
GL@MO
Enthusiast
 
Posts: 25
Liked: 2 times
Joined: Fri Nov 02, 2012 9:04 pm
Location: Vreden, Germany

Re: V7 - False alerts

Veeam Logoby Vitaliy S. » Wed Sep 04, 2013 6:42 pm

Hello Guido,

Did you have a chance to check the reason for this behavior with VMware support team? Still wondering if there is any pattern in this "weird" behavior.

Thank you!
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: V7 - False alerts

Veeam Logoby mcwill » Thu Sep 05, 2013 10:30 am

We are also seeing this, from what we've observed...

  • Alerts started after upgrading to 7.0 from 6.5. Now receiving approx. 3 alerts per day from 50 VMs
  • Latency also appears in VCenter performance current graph, but not in daily or any other rollup. May be filtered as part of roll-up process?
  • Latency alert always relates to a associated backup snapshot creation/removal
Backups are processed by hotadd and we have parallel processing enabled.
mcwill
Enthusiast
 
Posts: 64
Liked: 10 times
Joined: Sat Jan 16, 2010 9:47 am
Full Name: Iain McWilliams

Re: V7 - False alerts

Veeam Logoby Vitaliy S. » Thu Sep 05, 2013 10:39 am

Hi Iain,

1. You didn't see that in 6.5 cause we have added a couple of new predefined alarms to v7.
2. Yes, VMware rollups real-time performance data, so these spikes are not visible in the daily or other reporting periods.
3. I would recommend to involve VMware support team in the troubleshooting process, as these extremely high values for latency shouldn't be true.

Thank you!
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: V7 - False alerts

Veeam Logoby JROleon » Fri Sep 06, 2013 10:39 am

Hi,
I'm also experiencing the same problem. No case made. I'll keep an eye on this topic.
JROleon
Lurker
 
Posts: 1
Liked: never
Joined: Tue Oct 30, 2012 1:07 pm
Full Name: Jeroen Roegiers

Re: V7 - False alerts

Veeam Logoby Vitaliy S. » Fri Sep 06, 2013 1:04 pm

Jeroen, I would also suggest contacting VMware support on this, the more info they have the better.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

[MERGED] VM total disk latency

Veeam Logoby JosueM » Mon Sep 09, 2013 5:28 pm

Good day everyone,

I just installed veeam 7 no changes made on default config except the email notification, suddenly I started to receive a lot of emails for random VMs with VM total disk latency alert then in minutes the alert goes back to green, this happens more or les each 5 to 10 minutes for multiple vms.

I have noticed no alert is triggered on vcenter itself and no user is complaining about performance. Not sure how risky this message could be.

I attached a copy of the message
Image
image upload
JosueM
Expert
 
Posts: 147
Liked: 9 times
Joined: Sat Sep 01, 2012 2:53 pm
Full Name: Josue Maldonado

Re: V7 - False alerts

Veeam Logoby GL@MO » Mon Sep 09, 2013 6:48 pm

hmmmmm latency notifications are gone...
GL@MO
Enthusiast
 
Posts: 25
Liked: 2 times
Joined: Fri Nov 02, 2012 9:04 pm
Location: Vreden, Germany

Re: VM total disk latency

Veeam Logoby rfn » Mon Sep 09, 2013 6:54 pm

This is happening here as well. I've been told by support (Case # 00434916) that the cause is that Veeam One 6 didn't know about the counter "Total Disk Latency" which is supposed to be a fairly new counter. Veeam One 7 sees this counter and correctly flags it as an error when vCenter reports these high numbers.

Veeam support suggests that I contact VMware but I don't have a support agreement - only maintenance - so I might just disable this alarm since it looks to be a bug. It's happening when snapshots are being committed during backups.
rfn
Expert
 
Posts: 131
Liked: 4 times
Joined: Wed Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen

Re: V7 - False alerts

Veeam Logoby Vitaliy S. » Mon Sep 09, 2013 8:04 pm

Hi Josue.

Rene is correct, if you do not have any issues you may temporary disable this alarm and open a support case with VMware for further troubleshooting.

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: V7 - False alerts

Veeam Logoby JosueM » Mon Sep 09, 2013 8:12 pm

Ok, thanks for your help. Is just the message at first time seems like critic but after reading on this thread it seems more like a warning.
JosueM
Expert
 
Posts: 147
Liked: 9 times
Joined: Sat Sep 01, 2012 2:53 pm
Full Name: Josue Maldonado

Re: V7 - False alerts

Veeam Logoby KOMBillM » Wed Sep 11, 2013 1:00 pm

We are also seeing this. Every night I get an alarm for one specific VM:

"Datastore Highest Latency" (3353953467950 ms) is above a defined threshold (75 ms)

If you do the math I believe it works out to 106 years, which is absurd.
KOMBillM
Novice
 
Posts: 5
Liked: never
Joined: Tue Jan 03, 2012 9:14 pm
Full Name: Bill MacEachern

PreviousNext

Return to Veeam ONE



Who is online

Users browsing this forum: No registered users and 4 guests