VM total disk latency

[Pat490]

Since a while I do get constantly warnings from alarm "VM total disk latency" for 1 VM with "Datastore Highest Latency" in details > 200+ milliseconds

but when I check vCenter graphs I can not find those high numbers. Also in V1 Monitor virtual disks graph I do not see those number. ESXTOP is also low.

So I am wondering why the alarm is triggered?

[Shestakov]

Hello Patrick,
That may be caused by performance metrics averaging. How much time past between the alarm was triggered and you checked Veeam ONE Monitor graphs? What data collection mode are you at (for typical infrastructures or for advanced ones)?
Thanks!

[Pat490]

time between alarms varies... for example today it was 8:29, 9:18, 9:50, 10:25, 11:14, 11:49, 12:45, 14:51

screenshot in last post was from VeeamOne Monitor. Or did I missunderstand?

What data collection mode? what do you mean? Here are the alarm settings:

[Shestakov]

What I meant that depending on data collection mode, Veeam ONE collects 20-second samples from ESXi and shows it during the day or so(depends on the settings), then it aggregates 20-second values to 5-minutes intervals and shows the average value and so on up to 2-hour averages. These numbers are for Typical data collection mode.
According to your alarm settings, it will trigger if 15 min average Latency is higher than 75msec. If you check the performance graph a day after the alarm was triggered you may not find spikes as they are flattered.

What data collection mode? what do you mean?

I meant this setting. The one you choose during setup.

[Pat490]

It should be set to typical as we have small environment with only 3 hosts.
I am looking at graphs for a few days now and never saw peaks as high as written in the alarms with 200ms or more

[Shestakov]

Do you observe same issue with other alarms?
I would also recommend contact support for closer investigation.
Thanks!

[Pat490]

rarely I get a "Datastore read latency" alarm for 1 datastore which I also can not see in vSphere graphs

[Shestakov]

Got it, thanks!
If you decide to contact support team, please post your case number.

[veit]

Hello,

sorry for pushing up this topic again. Since we face the exact same issue, i wanted to avoid opening a dublicated thread.
We receive rathe often datastore alarms for read or wirte latencies. They fire up for different datastores at random times.
Our infrastructure consists of 2 hosts, each connected to 20 datastores. In Average we find every day 5 to 10 latency warnings for all datastores. So only a few datastores each day, but changing from day to day.
My aim is it to correlate the warnings with perfmormance data collected for the datastore. Warning / Alarm is set as 15min max above 100ms / 250ms.
The tiniest interval i can find in I/O Graph or in the raw-data report is a 5 Minute average. Those latency peaks never appear in the 5min average. And i find it impossible to get IOP/s information for the exact time when latency warning comes up.

Example. Datastore "vol03" fired up warning read latency on 31.08.2017 at 12:53 with 153ms. Raw Data shows "Value(Millisecond)" for Datastore read latency in interval 31.08.2017 12:55: 0ms. Next interval (31.08.2017 13:00) shows 47ms, but would be wrong interval regarding the time of the warning. Raw Data IOPs : 31.08.2017 12:50 = 43 and on 31.08.2017 IOPs are at 93. Which is rather low i suppose.

Is there a way to get more fine grained raw Data / Performance Data. IOPs max would be interesting. Or 1 minute average for latency.

[Shestakov]

Hello veit,
You can change performance data granularity via support team. Note that it will increase database size and may slightly slow down application.
As for raw data, check Raw Performance Data report.
Thanks!