Monitoring and reporting for Veeam Backup & Replication, VMware vSphere and Microsoft Hyper-V
Post Reply
mlange
Novice
Posts: 6
Liked: never
Joined: Sep 02, 2014 3:06 pm
Full Name: Matt Lange
Contact:

Odd latency reporting

Post by mlange » Dec 26, 2018 9:21 pm

I've been getting a lot of high datastore latency notifications recently, often reporting into the 100's of milliseconds. However we don't actually seem to be experiencing any latency of that nature. Looking at the datastores themselves the highest peaks I see are 20 ms, and those aren't even happening during the times Veeam One is reporting. We're running Veeam One 9.5 Update 3, ESXi 6.5 and the datastores are on a Nimble array. Is this a known issue or is there a patch or something I'm missing?

Vitaliy S.
Veeam Software
Posts: 21865
Liked: 1319 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Odd latency reporting

Post by Vitaliy S. » Dec 27, 2018 9:21 am

Hi Matt,

This is not a known issue, any metric reported by Veeam ONE is retrieved from a vCenter Server. Can you please post screenshots of these latency peaks taken from the vSphere Client and Monitor Client?

Thanks!

mlange
Novice
Posts: 6
Liked: never
Joined: Sep 02, 2014 3:06 pm
Full Name: Matt Lange
Contact:

Re: Odd latency reporting

Post by mlange » Dec 27, 2018 2:52 pm

Ok, so maybe I don't understand how any of this is supposed to work, I guess we'll find out.

Here is a notification email I received this morning:
Image

This is what VSphere shows:
Image

This is what the Nimble shows:
Image

This is what the Monitor shows for the VM in question:
Image

And what the Monitor shows for that datastore in general:
Image

I'm new to using VeeamOne, and I didn't set ours up so this could just be me not understanding how it supposed to work the way it is configured, or it could be misconfigured and I just don't understand where or why.

Shestakov
Veeam Software
Posts: 6167
Liked: 547 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Odd latency reporting

Post by Shestakov » Dec 31, 2018 2:19 pm

Hello Matt,
Do first two graphs represent data from 27dec?

mlange
Novice
Posts: 6
Liked: never
Joined: Sep 02, 2014 3:06 pm
Full Name: Matt Lange
Contact:

Re: Odd latency reporting

Post by mlange » Dec 31, 2018 2:28 pm

Yes, all of the images are from the 27th.

Shestakov
Veeam Software
Posts: 6167
Liked: 547 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Odd latency reporting

Post by Shestakov » Dec 31, 2018 2:36 pm

That doesn't look expected. I would suggest to contact Veeam support.
The only reason why alarm says 57msec, while Veeam ONE Monitor shows around 8msec is the data aggregation.

ViciousXUSMC
Novice
Posts: 8
Liked: never
Joined: Dec 03, 2018 7:57 pm
Full Name: Patrick Mosley
Contact:

Re: Odd latency reporting

Post by ViciousXUSMC » Jan 02, 2019 12:19 pm

I think I am having the same issue. When I first installed Veeam One I was inundated with latency notices.
I never had vCenter tell me anything was wrong, no users see anything wrong. I just figured I had not tuned my monitor yet and the default values were too strict.

I turned that alert off actually but may turn it back on again and see if I get the same crazy numbers.

Vitaliy S.
Veeam Software
Posts: 21865
Liked: 1319 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Odd latency reporting

Post by Vitaliy S. » Jan 04, 2019 9:16 am

Hi Patrick,

Turning off the alarm is not recommended, however, you can indeed tune the threshold to match your environment behavior. Go to the host or datastore tab and select the latency counter to be displayed on the chart. Once you do this, verify what is the average latency, add 10%/20% and set the new threshold in the alarm settings.

Thanks!

Shestakov
Veeam Software
Posts: 6167
Liked: 547 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Odd latency reporting

Post by Shestakov » Jan 05, 2019 4:17 pm

Also notice that the "normal latency" depends on hardware. For older datastores higher latency may be expected.
So you may want to set different thresholds for different datastores.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests