Monitoring and reporting for Veeam Backup & Replication, VMware vSphere and Microsoft Hyper-V
Post Reply
AlainRussell
Enthusiast
Posts: 49
Liked: 1 time
Joined: Aug 27, 2011 12:04 am
Full Name: Alain Russell
Contact:

V7 - VMware Issue - False alerts

Post by AlainRussell » Aug 18, 2013 8:32 pm

Since upgrading to v7 I've been getting some false alerts as below, I can't see a corresponding warning in Vcenter so not sure where to look to fix? Any ideas?

"Datastore Highest Latency" (28251344 ms) is above a defined threshold (75 ms)"

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: V7 - False alerts

Post by Vitaliy S. » Aug 18, 2013 8:41 pm

Hi Alain,

What values are displayed on the performance graphs in the vSphere Client for this metric? Can you please open a support case with our technical team and let them have a look at your debug logs?

Thanks!

AlainRussell
Enthusiast
Posts: 49
Liked: 1 time
Joined: Aug 27, 2011 12:04 am
Full Name: Alain Russell
Contact:

Re: V7 - False alerts

Post by AlainRussell » Aug 19, 2013 1:27 am

There are no alerts showing (screenshot attached), strange that the false alert coincides with the Backup job completing. I'm travelling at the moment but will open a support case when I get back - not a major issue currently.

http://utility.blackpepper.co.nz/alain/VCenter.png
http://utility.blackpepper.co.nz/alain/Alert.png
http://utility.blackpepper.co.nz/alain/Datastore.png

Datastore latency shows a max of 19ms read, 16ms max over the last 2 hours.

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: V7 - False alerts

Post by Vitaliy S. » Aug 19, 2013 9:15 am

Ok, let me check with our QC team if it is a known issue or not.

emsaus
Novice
Posts: 3
Liked: never
Joined: Aug 14, 2012 2:08 am
Full Name: Ben Howell
Contact:

Re: V7 - False alerts

Post by emsaus » Aug 19, 2013 11:00 pm

We are also seeing the same thing. Upgraded to V7 yesterday the next backup that ran generated the same alarm. "Datastore Highest Latency (3869081 ms) is above a defined threshold (75 ms)". It lasted for about 20 mins then cleared however the backup took ~60mins to run. No mention of any issues in vcenter server.

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: V7 - False alerts

Post by Vitaliy S. » Aug 20, 2013 8:02 am

Ben, can you also open a support case with our technical team and post your ticket number, so I could discuss it with our dev team?

Jadawyn
Lurker
Posts: 1
Liked: never
Joined: Aug 31, 2012 8:41 am
Full Name: Luca Bedogni
Contact:

Re: V7 - False alerts

Post by Jadawyn » Aug 20, 2013 10:50 am

Same problem here. It only happens when Backup job completes. No warning from vcenter.

emsaus
Novice
Posts: 3
Liked: never
Joined: Aug 14, 2012 2:08 am
Full Name: Ben Howell
Contact:

Re: V7 - False alerts

Post by emsaus » Aug 21, 2013 5:26 am

Vitaliy S. wrote:Ben, can you also open a support case with our technical team and post your ticket number, so I could discuss it with our dev team?
No problems, case id: 00432028

Thanks.

rfn
Expert
Posts: 131
Liked: 4 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: V7 - False alerts

Post by rfn » Aug 25, 2013 8:30 pm

I'm having this problem as well... I haven't created a case yet.

TomBlue
Influencer
Posts: 15
Liked: 3 times
Joined: Mar 23, 2012 8:19 am
Full Name: Thomas Blühmann
Contact:

Re: V7 - False alerts

Post by TomBlue » Aug 27, 2013 8:50 am

We exoperience the same issue. I opened a case: 00434907

rfn
Expert
Posts: 131
Liked: 4 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: V7 - False alerts

Post by rfn » Aug 27, 2013 9:11 am

I now has a case as well: Case # 00434916

Orven
Lurker
Posts: 1
Liked: never
Joined: Aug 27, 2013 12:29 pm
Full Name: Olaf Reitz
Contact:

Re: V7 - False alerts

Post by Orven » Aug 27, 2013 12:31 pm

Same issue since Update to v7 case # 00435011

TomBlue
Influencer
Posts: 15
Liked: 3 times
Joined: Mar 23, 2012 8:19 am
Full Name: Thomas Blühmann
Contact:

Re: V7 - False alerts

Post by TomBlue » Sep 02, 2013 7:26 am

TomBlue wrote:We exoperience the same issue. I opened a case: 00434907
UPDATE:

We investigated our case a little deeper, thanks to Veeam Support. We recognized that the spikes we experience in Veeam One are reported as well in vSphere performance monitor. Here is one example where you can see this spikes:

http://sdrv.ms/1ad6cMV (Veeam One: Datastore Highest Latency)
http://sdrv.ms/1ad6kMx (vSphere: VM Disk Highest Latency)
http://sdrv.ms/1ad6raR (vSphere: Datastore Highest Latency)

I cannot find any jobs that can cause these extrem spikes. If there were real latencies like this, I guess we would experience performance issues in any application. As well as the "Datastore Highest Latency" shows negative numbers, strange isn't it? I just found out that around the same time as the high latency is occuring, the machine in question has high CPU load, but not sure yet if this causes the high latency or if hight latency is the reason for CPU load.

We do not have any solution for this yet, but I'm curious if anyone has the same experience and maybe have a solution on this. Of course I agree to Veeam support that this is a problem in vSphere, not in Veeam One, though these error started with update of Veem One + Veeam B&R to version 7.0.

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1380 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: V7 - False alerts

Post by Vitaliy S. » Sep 02, 2013 11:25 am

Thomas,

Yes, our R&D team has confirmed that the same spikes are observed in the vSphere Client as well, thanks for the update.

Btw, do you experience these issues at the particular backup job operation? For example, during snapshot commit operation or similar? What processing mode do you use for your backup jobs? HotAdd?

Thanks!

TomBlue
Influencer
Posts: 15
Liked: 3 times
Joined: Mar 23, 2012 8:19 am
Full Name: Thomas Blühmann
Contact:

Re: V7 - False alerts

Post by TomBlue » Sep 03, 2013 6:06 pm

Hi Vitaly!
No, in my case there is no relation to the backup jobs. I do backups only once a night and spikes I can see about 1 - 3 times per hour on this specific VM. On two other VMs I get the error only 1 or 2 times a day. There is also no connection to any jobs on these servers. I still do not have any idea.

Using network mode for backup. Veeam B&R and Veeam One is installed on a physical machine.

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests