Real-time performance monitoring and troubleshooting
tym
Enthusiast
Posts: 83
Liked: 9 times
Joined: Mar 26, 2015 9:30 pm
Full Name: Tim Diekhans
Contact:

VM disk errors

Post by tym »

Hi,

I am receiving Veeam One disk Errors from time to time:
Object:M09
Object Type:Hyper-V VM
Location:HyperV-Cluster02.mydomain.com / Resources
Host:v05.mydomain.com
Status:Error
Alarm:VM disk errors
Time:22.09.2015 16:39:37
Details:"Errors/min" (15) is above a defined threshold (8)

KB article
Summary:This VM has logged one or more errors that have occurred on its virtual device
Cause:The error count should always be zero for the virtual storage device
Resolution:Check the status of the disk subsystem and connectivity. Check for possible disk hardware, firmware or driver issues. See historical graph of error counts


This error is just visible in VO, I cannot find any hints regarding this message neither on the vms nor on the cluster hosts. Everything seems to run smooth, but I am nevertheless concerned, where these error messages are coming from?

Best,
Tim
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: VM disk errors

Post by PTide »

Hi,
This error is just visible in VO, I cannot find any hints regarding this message neither on the vms nor on the cluster hosts.
Have you checked Event Viewers in guest OS and on Hyper-V host? Also please check HV host's disk subsystem with a perfmon. The idea behind my suggestion is that Veeam One does not create events and errors at will - it just processes data that is coming from somewhere (HV host for example).If there are no traces of error in guest OS logs and Hyper-V host logs, and perfmon does not show that disks are under heavy usage then that might be a false positive. In such case please open a ticket with support and post your case ID here.

Thank you.
tym
Enthusiast
Posts: 83
Liked: 9 times
Joined: Mar 26, 2015 9:30 pm
Full Name: Tim Diekhans
Contact:

Re: VM disk errors

Post by tym »

ok, I´ll open a ticket. Actually, I found no other error messages on the guest or host os event logs during this time window.
tym
Enthusiast
Posts: 83
Liked: 9 times
Joined: Mar 26, 2015 9:30 pm
Full Name: Tim Diekhans
Contact:

Re: VM disk errors

Post by tym »

case id: 01069777
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM disk errors

Post by Vitaliy S. »

These errors should be displayed in perfmon, so I would check that as well.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM disk errors

Post by Shestakov »

To be exact, you need to check "Error Count" metric of "Hyper-V Virtual Storage Device" pack.
The alarm counts the number of errors happening and triggers the alarm if threshold is reached.
Thanks!
JoshRountree
Enthusiast
Posts: 34
Liked: 2 times
Joined: Feb 23, 2012 5:49 pm
Full Name: Josh Rountree
Contact:

Re: VM Disk Errors on 2012

Post by JoshRountree »

I'm seeing these same errors through PerfMon - I think it's possibly related to having to add a SCSI controller to the VM so it won't hang during backup operations but I can't confirm this.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM Disk Errors on 2012

Post by Vitaliy S. »

Yes, it is always a good idea to compare this counter with perfmon. Good idea, may we would be able to verify this internally.
JoshRountree
Enthusiast
Posts: 34
Liked: 2 times
Joined: Feb 23, 2012 5:49 pm
Full Name: Josh Rountree
Contact:

Re: VM Disk Errors on 2012

Post by JoshRountree »

That would be awesome if you can confirm. I feel pretty strongly it's a false positive, but if it can be confirmed it would make me feel better.
frenchviking
Lurker
Posts: 1
Liked: never
Joined: Sep 02, 2016 7:04 am
Contact:

[MERGED] VM disk error

Post by frenchviking »

Hello,

My environment is 2 Windows 2012 R2 Hyper-V hosts. For all VMs all integrations services are activated. My storage appliance is a Dell MD3820i, through iSCSI I present 3 LUNs to the hosts.

I have multiple VMs reported with disk errors. All VMs with errors are on the same LUN and all VMs are Windows 2012 R2. On the same LUN Linux (CentOS) VMs do not report issue. Others VMs on the two other LUNs do not have disk error reported. I can't tell if the problems come from the Windows guest or from the LUN.
Reported issue is the following :
Alarm:VM disk errors
Time:09/09/2016 11:06:21
Details:"Errors/min" (16.0 Number) is above a defined threshold (8.0 Number)

KB article
Summary:This VM has logged one or more errors that have occurred on its virtual device
Cause:The error count should always be zero for the virtual storage device
Resolution:Check the status of the disk subsystem and connectivity. Check for possible disk hardware, firmware or driver issues. See historical graph of error counts
I don't encounter any issue within the VMs, no error in the event viewer.

Any Ideas ?

Thank you.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM disk errors

Post by Shestakov »

Hello and welcome to the community!
As stated above Veeam ONE alarm use Hyper-V counter called Error Count which "represents the total number of errors that have occurred on the virtual device".
It`s a generic error so I would review logs and/or contact Microsoft support.
You may also check if the alarm is triggered properly by checking the mentioned counter in Perfmon.
Thanks!
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: VM disk errors

Post by Shestakov »

One of the possible reasons is use of SCSI attached Pass-through disk.
Errors occur because the guest OS sends a SCSI command which is not supported by Hyper-V in this scenario. As a result, the host OS increments the Error Count performance counter.
Microsoft states this is expected behavior and no corrective action is required. However you should monitor the performance of the Pass-through disk from within the guest OS.
Thanks!
pjw2020
Novice
Posts: 7
Liked: never
Joined: Jan 20, 2020 10:04 am
Full Name: Paul Webb
Contact:

Re: VM disk errors

Post by pjw2020 »

Did anyone ever get a conclusive answer on this topic as I have a handful of VMs that typically go over threshold once a day. When I dskcheck inside the VM no errors are reported. The Cluster Doesn't give any alerts.
The term "Error/Min" is vague - can anyone expand on what this actually means. Assue these are retryable "errors" as we would be experiencing data corruption if there were actual disk errors occuring.
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: VM disk errors

Post by wishr »

Hi Paul,

Have you checked the aforementioned metric on the Hyper-V side?

Thanks
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: VM Disk Errors on 2012

Post by Vitaliy S. »

Hi Paul,

There should not be data corruptions, I have merged your thread with links to Microsoft articles talking about when these errors can go up. Briefly take a look at those, and if that's not your case change the threshold to a higher value and keep monitoring your VMs.

Thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 7 guests