Question about VM total disk latency

Monitoring and reporting for Veeam Backup & Replication, VMware vSphere and Microsoft Hyper-V

Question about VM total disk latency

Veeam Logoby DonnieBrasco » Wed Mar 16, 2016 3:24 am

Hi

I just installed VEEAM one in our setup and all the alarms are configured with default settings.

I have been getting VM total disk latency warning and errors for many VMs. When I go to the vCenter server performance charts and look at vmdisk latency stats, I can see that there is latency spike at the time when warning/alert was generated in VEEAM one.

My question is do we have any threshold before we raise warning or alert in VEEAM one or VEEAM one will keep generating warning/alerts even if there is single I/O latency spike observed in vCenter server for that particular vmdisk.

-NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Wed Mar 16, 2016 12:01 pm

Hi,
Yes, there is a Time period option to set the interval for the alarm. So the alarm warns you if average performance during the interval is above the threshold.
Thanks!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby Vitaliy S. » Wed Mar 16, 2016 2:46 pm

The default for most alarms is 15 minutes, so you don't need to tweak anything and will only be alerted when there is a high load for an extended period of time.
Vitaliy S.
Veeam Software
 
Posts: 19564
Liked: 1104 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Wed Mar 16, 2016 3:42 pm

Thanks for your response.

If I go to alarm settings, I can see that there under rules tab, we have value time period setting and default value is 15 min.

Now, does that mean that alarm will be generated when average latency for that VM goes beyond 50ms for last 15 mins or latency for the VM was above 50ms for 15 mins ?

-NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Wed Mar 16, 2016 3:48 pm

The alarm will be generated when average latency for that VM goes beyond 50ms for last 15 mins
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Wed Mar 16, 2016 3:53 pm

Thanks Nikita for your response.

If VEEAM one checks for the average latency for last 15 mins, they why when the alarm is triggered the latency no. shown for the alarm is not the average latency no. but it is the latest latency spike seen in the vCenter performance graph for a vmdisk ?

Also, as per my understanding, VEEAM one polls data from vCenter server on realtime basis and vCenter real time performance charts are updated every 20 secs so does that mean that VEEAM one does the analytic and computes 15 min average latency and then shows the alert ?

-NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Wed Mar 16, 2016 4:14 pm 1 person likes this post

Veeam ONE collects 20-second samples from ESXi and shows it during the day or so(depends on the settings), then it aggregates 20-second values to 5-minutes intervals and shows the average value and so on up to 2-hour averages.
For 15-minutes interval Veeam ONE takes 20 sec samples over 15 minutes and calculates the average value.
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Wed Mar 16, 2016 4:42 pm

Just one more thing I need to understand

For example - In vCenter server real time performance graph there was a latency spike of 100ms and after that it comes back to normal.
Now in VEEAM one, in case there is a warning generated for the same VM, I don't see the warning generated with average latency no. It will generate alert

"Datastore Highest Latency" (100.0 Milliseconds) is above a defined threshold (75.0 Milliseconds)"

Can someone explain me why we are reporting highest spike which was observed by the VM and not the average latency ?

-NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby v.tom » Thu Mar 17, 2016 1:01 pm

We are observing this problem also in our lab.
It triggers the alarm on a one time spike, not the average in the specified time period.

vCenter 6.0 build 3018523
ESXi 5.5 build 3116895
Veeam ONE 9.0.0.2062
VMCE certified
v.tom
Influencer
 
Posts: 22
Liked: 2 times
Joined: Tue May 06, 2014 2:28 pm
Full Name: Tom

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Thu Mar 17, 2016 3:25 pm

That`s a known issue for that particular alarm. Our R&D team is working on a fix.
You may contact technical support to get the fix as soon as it`s available.
Thanks!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Thu Mar 17, 2016 3:28 pm

Hello Nikita,

Do you have any Bug ID for the reference and also the exact problem description ?

-NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Thu Mar 17, 2016 3:32 pm

They should know the issue.
You can link them to this forum topic.
Thanks!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Thu Mar 17, 2016 3:36 pm

I did. They asked me to upload the logs.
I don't think logs are relevant for this case.

Case #01732408
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Re: Question about VM total disk latency

Veeam Logoby Shestakov » Thu Mar 17, 2016 3:45 pm

Well, there is still a probability you have a different issue. So they need to make sure.
You may forward them the reference to an original support ticket 01686624 and the bug is tracked as Issue 68288
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Question about VM total disk latency

Veeam Logoby DonnieBrasco » Thu Mar 17, 2016 4:54 pm

Hello Nikita,

Is it specific to some ESXi or vCenter server version ? Or it is a issue independent of vCenter/ESX version ?

NP
DonnieBrasco
Novice
 
Posts: 7
Liked: never
Joined: Wed Mar 16, 2016 2:11 am
Full Name: Navpreet Rakwal

Next

Return to Veeam ONE



Who is online

Users browsing this forum: No registered users and 5 guests