Real-time performance monitoring and troubleshooting
WinstonWolf
Expert
Posts: 187
Liked: 4 times
Joined: Jan 06, 2011 8:33 am
Contact:

Datastore Write Latency much too High from VeeamOne !!!

Post by WinstonWolf » Dec 06, 2016 8:27 am

Hello ,
I have every Morning 60 Mails with Datastore Write Latency Warning or Error Messages from Veeam One Site .
Like this :
Image

But on the other Side we use the Netapp VA OCPM ( OnCommand Performance Manager ) & the OCUM ( On Command Unified Manager )
This are special VAs from Netapp which Monitor the Datastores and Volumes also. And sorry Veeam, i trust this Netapp own tools more then Veeam One ;-)

And the Output from this Tools is for the same Datastore RZ1_vmware_ds10 this :
Image

As you can see the Disk write latency is only on the highest peaks and only for an short period 20.0 ms .

What Veeam One see there ??!?!?

Any Statement for this ?

Shestakov
Veeam Software
Posts: 6409
Liked: 611 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Shestakov » Dec 06, 2016 8:54 am

Hello,
Could you check what type of aggregation do you have set up in the triggered alarm(Avg/Max)?
I believe that`s why there are different values.

Usually monitoring tools have to aggregate data in 5 or 20 minutes averages and products may have different time frames.
Have you checked this metric in Veeam ONE Monitor?

Thanks!

Mike Resseler
Product Manager
Posts: 5343
Liked: 560 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Mike Resseler » Dec 06, 2016 9:37 am

Hi Winston,

Could you compare the data from Veeam ONE also with the data you see in vCenter? Because that is where we grab our information. I am not sure (but don't know either so ;-)) that we are comparing apples with apples here. What metric are you looking at with OCMP? (I don't know what MS / OP means and it might be a different metric.)

Thanks
Mike

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1382 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Vitaliy S. » Dec 07, 2016 4:44 pm

Yes, Mike is correct. Veeam ONE cannot make up data, we collect it via vSphere API, so if there is a spike reported in Veeam ONE, it will also be reflected in the vSphere Client.

WinstonWolf
Expert
Posts: 187
Liked: 4 times
Joined: Jan 06, 2011 8:33 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by WinstonWolf » Dec 08, 2016 8:39 am

Hello again ,

Here is the Screenshot from my Alarm Rules from "Datastore Write Latency"
Image

Can it be better with Aggregation : "Avg" ?

Here is an Screenshot from Write Latency from the Vcenter Server , as you can see there are some Spikes but not for long time ( they can produced by Storage Snapshots maybe )

Image

That means for me to Switch the default Setting from Aggregation max to Aggregation Avg . or ?

Thanks
Michael

Thanks
Michael

WinstonWolf
Expert
Posts: 187
Liked: 4 times
Joined: Jan 06, 2011 8:33 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by WinstonWolf » Dec 08, 2016 9:13 am

Hello ,

Now i think Veeam One runs totaly wrong .
At the Moment i become this Warning ( Error ) Mail from Veeam One which told me as follows :

Image

But when i look on my Vcenter Server the the Performance Graph from this Datastore shows this :

Image

Please Veeam , can u explain it for me .

Thanks

Shestakov
Veeam Software
Posts: 6409
Liked: 611 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Shestakov » Dec 08, 2016 11:07 am

Hello Michael,

The alarm has been triggered because you`ve set aggregation=max. So once you had a spike higher than the threshold, alarm worked.
Regarding the last screenshot, could you also check values in Veeam ONE Monitor for the same time frame? If you think that Veeam ONE doesn`t work correctly, please contact Veeam support team for the detailed investigation. The will check if aggregation, object and metric are chosen correctly. Once you open a case, please let us know your case ID.

Thanks!

WinstonWolf
Expert
Posts: 187
Liked: 4 times
Joined: Jan 06, 2011 8:33 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by WinstonWolf » Dec 08, 2016 12:09 pm

My First Stepp now is to completely uninstall and reinstall Veeam One . I also delete the Veeam One Database .
Then i observe if the Events comes again . Its not so much work ;-)

Shestakov
Veeam Software
Posts: 6409
Liked: 611 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Shestakov » Dec 08, 2016 12:35 pm

Well... Sounds more time-consuming than a call to the support team :)

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1382 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Vitaliy S. » Dec 08, 2016 12:37 pm

And re-installing the data collection engine cannot fix this (if the issue exists) ;)

bsylvain
Novice
Posts: 5
Liked: 3 times
Joined: Sep 18, 2014 11:28 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by bsylvain » Jan 06, 2017 3:03 pm

I have the same "inconvenience" from Veeam One.

MAX seems to be the default parameter for this alarm (read/write latency).
I tried to switch to avg but it doesn't seems to keep the setting (if I go back in the alarm definition it's back to MAX), I have to create a custom alarm and disable the defaults ones. Is this normal?

Thanks.
Sylvain

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1382 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Vitaliy S. » Jan 06, 2017 3:05 pm 1 person likes this post

Hi Sylvain,

If the new setting is not preserved in the alarms configuration, then please open a support ticket and let our team take a look at this. Based on your description, it sound like a bug.

EDIT: Just checked it with our support team and they have a fix for this problem. Please ask for Update 1 RC to fix this issue.

Thanks!

bsylvain
Novice
Posts: 5
Liked: 3 times
Joined: Sep 18, 2014 11:28 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by bsylvain » Jan 06, 2017 3:14 pm

Hi Vitaliy,

Thanks, I will wait for official release then, no hurry.

Vitaliy S.
Product Manager
Posts: 22127
Liked: 1382 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by Vitaliy S. » Jan 06, 2017 3:15 pm

Ok, it should be a long wait. We are aiming to make it GA soon.

bsylvain
Novice
Posts: 5
Liked: 3 times
Joined: Sep 18, 2014 11:28 am
Contact:

Re: Datastore Write Latency much too High from VeeamOne !!!

Post by bsylvain » Feb 17, 2017 2:20 pm 3 people like this post

Hi

I forgot to follow up, but of course the update 1 fixed the problem. I have been able to set the alarms to AVG, no more annoyance.

Thanks

Sylvain

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest