Datastore Write Latency much too High from VeeamOne !!!

Real-time performance monitoring and troubleshooting

Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby WinstonWolf » Tue Dec 06, 2016 8:27 am

Hello ,
I have every Morning 60 Mails with Datastore Write Latency Warning or Error Messages from Veeam One Site .
Like this :
Image

But on the other Side we use the Netapp VA OCPM ( OnCommand Performance Manager ) & the OCUM ( On Command Unified Manager )
This are special VAs from Netapp which Monitor the Datastores and Volumes also. And sorry Veeam, i trust this Netapp own tools more then Veeam One ;-)

And the Output from this Tools is for the same Datastore RZ1_vmware_ds10 this :
Image

As you can see the Disk write latency is only on the highest peaks and only for an short period 20.0 ms .

What Veeam One see there ??!?!?

Any Statement for this ?
WinstonWolf
Expert
 
Posts: 181
Liked: 4 times
Joined: Thu Jan 06, 2011 8:33 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Shestakov » Tue Dec 06, 2016 8:54 am

Hello,
Could you check what type of aggregation do you have set up in the triggered alarm(Avg/Max)?
I believe that`s why there are different values.

Usually monitoring tools have to aggregate data in 5 or 20 minutes averages and products may have different time frames.
Have you checked this metric in Veeam ONE Monitor?

Thanks!
Shestakov
Veeam Software
 
Posts: 5017
Liked: 417 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Mike Resseler » Tue Dec 06, 2016 9:37 am

Hi Winston,

Could you compare the data from Veeam ONE also with the data you see in vCenter? Because that is where we grab our information. I am not sure (but don't know either so ;-)) that we are comparing apples with apples here. What metric are you looking at with OCMP? (I don't know what MS / OP means and it might be a different metric.)

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 3342
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Vitaliy S. » Wed Dec 07, 2016 4:44 pm

Yes, Mike is correct. Veeam ONE cannot make up data, we collect it via vSphere API, so if there is a spike reported in Veeam ONE, it will also be reflected in the vSphere Client.
Vitaliy S.
Veeam Software
 
Posts: 19770
Liked: 1120 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby WinstonWolf » Thu Dec 08, 2016 8:39 am

Hello again ,

Here is the Screenshot from my Alarm Rules from "Datastore Write Latency"
Image

Can it be better with Aggregation : "Avg" ?

Here is an Screenshot from Write Latency from the Vcenter Server , as you can see there are some Spikes but not for long time ( they can produced by Storage Snapshots maybe )

Image

That means for me to Switch the default Setting from Aggregation max to Aggregation Avg . or ?

Thanks
Michael

Thanks
Michael
WinstonWolf
Expert
 
Posts: 181
Liked: 4 times
Joined: Thu Jan 06, 2011 8:33 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby WinstonWolf » Thu Dec 08, 2016 9:13 am

Hello ,

Now i think Veeam One runs totaly wrong .
At the Moment i become this Warning ( Error ) Mail from Veeam One which told me as follows :

Image

But when i look on my Vcenter Server the the Performance Graph from this Datastore shows this :

Image

Please Veeam , can u explain it for me .

Thanks
WinstonWolf
Expert
 
Posts: 181
Liked: 4 times
Joined: Thu Jan 06, 2011 8:33 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Shestakov » Thu Dec 08, 2016 11:07 am

Hello Michael,

The alarm has been triggered because you`ve set aggregation=max. So once you had a spike higher than the threshold, alarm worked.
Regarding the last screenshot, could you also check values in Veeam ONE Monitor for the same time frame? If you think that Veeam ONE doesn`t work correctly, please contact Veeam support team for the detailed investigation. The will check if aggregation, object and metric are chosen correctly. Once you open a case, please let us know your case ID.

Thanks!
Shestakov
Veeam Software
 
Posts: 5017
Liked: 417 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby WinstonWolf » Thu Dec 08, 2016 12:09 pm

My First Stepp now is to completely uninstall and reinstall Veeam One . I also delete the Veeam One Database .
Then i observe if the Events comes again . Its not so much work ;-)
WinstonWolf
Expert
 
Posts: 181
Liked: 4 times
Joined: Thu Jan 06, 2011 8:33 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Shestakov » Thu Dec 08, 2016 12:35 pm

Well... Sounds more time-consuming than a call to the support team :)
Shestakov
Veeam Software
 
Posts: 5017
Liked: 417 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Vitaliy S. » Thu Dec 08, 2016 12:37 pm

And re-installing the data collection engine cannot fix this (if the issue exists) ;)
Vitaliy S.
Veeam Software
 
Posts: 19770
Liked: 1120 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby bsylvain » Fri Jan 06, 2017 3:03 pm

I have the same "inconvenience" from Veeam One.

MAX seems to be the default parameter for this alarm (read/write latency).
I tried to switch to avg but it doesn't seems to keep the setting (if I go back in the alarm definition it's back to MAX), I have to create a custom alarm and disable the defaults ones. Is this normal?

Thanks.
Sylvain
bsylvain
Novice
 
Posts: 5
Liked: 3 times
Joined: Thu Sep 18, 2014 11:28 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Vitaliy S. » Fri Jan 06, 2017 3:05 pm 1 person likes this post

Hi Sylvain,

If the new setting is not preserved in the alarms configuration, then please open a support ticket and let our team take a look at this. Based on your description, it sound like a bug.

EDIT: Just checked it with our support team and they have a fix for this problem. Please ask for Update 1 RC to fix this issue.

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19770
Liked: 1120 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby bsylvain » Fri Jan 06, 2017 3:14 pm

Hi Vitaliy,

Thanks, I will wait for official release then, no hurry.
bsylvain
Novice
 
Posts: 5
Liked: 3 times
Joined: Thu Sep 18, 2014 11:28 am

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby Vitaliy S. » Fri Jan 06, 2017 3:15 pm

Ok, it should be a long wait. We are aiming to make it GA soon.
Vitaliy S.
Veeam Software
 
Posts: 19770
Liked: 1120 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Datastore Write Latency much too High from VeeamOne !!!

Veeam Logoby bsylvain » Fri Feb 17, 2017 2:20 pm 3 people like this post

Hi

I forgot to follow up, but of course the update 1 fixed the problem. I have been able to set the alarms to AVG, no more annoyance.

Thanks

Sylvain
bsylvain
Novice
 
Posts: 5
Liked: 3 times
Joined: Thu Sep 18, 2014 11:28 am


Return to Monitoring



Who is online

Users browsing this forum: No registered users and 2 guests