Real-time performance monitoring and troubleshooting
Post Reply
alubel
Influencer
Posts: 10
Liked: never
Joined: Aug 12, 2009 5:05 pm
Full Name: Andy
Contact:

v5 false alarm "Host hardware sensor status changed"

Post by alubel »

We have updated 2 of our ESX hosts and firmware to latest version a few weeks ago (4.1 260247) along with monitor v5. These are identically configured sun X4150's. within the vsphere console both hosts report "hardware status" is normal. Within the monitor v5 console, one of the hosts has an alarm going off every 10 minutes with alarm detail stating "Sensor VMware Rollup Health State equal Unknown" and in the hardware tab showing "VMware rollup health state" with a question mark. I have tried resetting the sensors on the affected host with no resolution.

Is there anymore information I could provide that would help in resolving this?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Hello Andy, it would be helpful, if you could look through a VeeamDCS.log file and see if there are any errors related to the second ESX host. Besides, I believe we may also check the sensor's status with Managed Object Browser to compare the behavior... Thanks!
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Andy, please use the instructions I've sent you via PM to check the sensor state with MOB. I would appreciate if you could post your results back here. Thanks!
alubel
Influencer
Posts: 10
Liked: never
Joined: Aug 12, 2009 5:05 pm
Full Name: Andy
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by alubel »

both are exactly the same, thanks for the advice - any other idea what could cause this?

https://host1/mob/?moid=ha-host&doPath= ... SensorInfo

Code: Select all

aseUnits	string	""
currentReading	long	0
dynamicProperty	DynamicProperty[]	Unset
dynamicType	string	Unset
healthState	ElementDescription	
NAME	TYPE	VALUE
dynamicProperty	DynamicProperty[]	Unset
dynamicType	string	Unset
key	string	"green"
label	string	"Green"
summary	string	"Sensor is operating under normal conditions"
name	string	"VMware Rollup Health State"
rateUnits	string	Unset
sensorType	string	"system"
unitModifier	int	0
https://host2/mob/?moid=ha-host&doPath= ... SensorInfo

Code: Select all

baseUnits	string	""
currentReading	long	0
dynamicProperty	DynamicProperty[]	Unset
dynamicType	string	Unset
healthState	ElementDescription	
NAME	TYPE	VALUE
dynamicProperty	DynamicProperty[]	Unset
dynamicType	string	Unset
key	string	"green"
label	string	"Green"
summary	string	"Sensor is operating under normal conditions"
name	string	"VMware Rollup Health State"
rateUnits	string	Unset
sensorType	string	"system"
unitModifier	int	0
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

For future readers: this is vCenter/ESX hardware interop issue that can be confirmed by browsing vCenter Server's MOB, which also displays unknown state for the VMware Rollup Health state hardware sensor. This is where Veeam Monitor obtains its data from.
jchampion01
Novice
Posts: 6
Liked: never
Joined: Nov 17, 2010 6:07 pm
Full Name: Jeremy
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by jchampion01 »

I am experiencing the same thing on my Dell R710 Servers 2 of the 3 are just fine, but my first one keeps having the Host hardware sensor status changed, for VMware Rollup Health and Drive Bay X Parity Check in progress. In the VI client the hardware status is fine on this drive array and host. Also physically there is little to no activty on the drives and the PERC card has not alerted me of any drive/array parity checks.
I have check the MOB for the VMware Rollup Health Status and it shows, current reading 0, Unset, and green status. Also the sensors for the hard drives and they raid array all show "green" status in MOB and VI CLient. Veeam Monitor keeps reporting this as an issue every 10 minutes.

Was there a resolution to this or, just live with false alarms? Which seems to defeat the purpose of monitoring...
Am i missing something here.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Hello Jeremy, could you please clarify whether you were using vCenter Sever MOB page or ESX host's one? The information about VMware Rollup Health State might be different in those cases.
jchampion01
Novice
Posts: 6
Liked: never
Joined: Nov 17, 2010 6:07 pm
Full Name: Jeremy
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by jchampion01 »

I was using the MOB page on the ESXi host, not the vCenter Server. I will check the vCenter Server MOB page and see if I can find my way around to check the sensor settings and values.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Yes, please do, because the issue we had with Andy related to vCenter server displaying incorrect status for this particular sensor, even though ESX host was reporting correctly.
jchampion01
Novice
Posts: 6
Liked: never
Joined: Nov 17, 2010 6:07 pm
Full Name: Jeremy
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by jchampion01 »

Could you tell me what links / objects i need to click on to drill down into that information, or the url path?
jchampion01
Novice
Posts: 6
Liked: never
Joined: Nov 17, 2010 6:07 pm
Full Name: Jeremy
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by jchampion01 »

Alright i think i found it on the vCenter Server MOB page.
It looks like this host (esxi01) on this vCenter Server MOB does show "Yellow" as the status for "VMware Rollup Health" and for the the RAID Array drives under "Drive X Parity Check In Progress - Assert".

VMware Rollup Health
https://shvcenter01/mob/?moid=host%2d20 ... SensorInfo

Code: Select all

[11] HostNumericSensorInfo Name Type Value 
baseUnits string "" 
currentReading long 0 
dynamicProperty DynamicProperty[] Unset 
dynamicType string Unset 
healthState ElementDescription Name Type Value 
dynamicProperty DynamicProperty[] Unset 
dynamicType string Unset 
key string "yellow" 
label string "Yellow" 
summary string "Sensor is operating under conditions that are non-critical" 
 
name string "VMware Rollup Health State" 
rateUnits string Unset 
sensorType string "system" 
unitModifier int 0 
 
Drive Parity Check
https://shvcenter01/mob/?moid=host%2d20 ... SensorInfo

Code: Select all

[131] HostNumericSensorInfo Name Type Value 
baseUnits string "" 
currentReading long 0 
dynamicProperty DynamicProperty[] Unset 
dynamicType string Unset 
healthState ElementDescription Name Type Value 
dynamicProperty DynamicProperty[] Unset 
dynamicType string Unset 
key string "yellow" 
label string "Yellow" 
summary string "Sensor is operating under conditions that are non-critical" 
 
name string "Disk Drive Bay 1 Drive 4: Parity Check In Progress - Assert" 
rateUnits string Unset 
sensorType string "Storage" 
unitModifier int 0 
 
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Jeremy, that's good that we were able to nail it. The resolution provided by Andy was the following, you may also try it, should help:
Andy wrote:Issue resolved by:
1. disable EVC on cluster.
2. vmotion machines over to secondary node.
3. maintenance mode / evict "faulted" node from cluster.
4. remove "faulted" node from vcenter.
5. log into "faulted" node via ILOM, restart management agents
6. re-add node back into vcenter.
7. re-add node to cluster.
8. re-enable EVC.
Thanks,
Vitaliy
Jpurdy79
Influencer
Posts: 17
Liked: never
Joined: Jun 18, 2010 1:46 pm
Full Name: Jonathan Purdy
Location: Franklin, TN

Re: v5 false alarm "Host hardware sensor status changed"

Post by Jpurdy79 »

I'm receiving the same alarms since my upgrade to 4.1. How do I access the sensor states in the Managed Object Browser?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Hello Jonathan, here you go:

1. Connect to your vCenter Server via Internet browser - https://vCenter_Server_name/mob/
2. Choose content ServiceContent content
3. Navigate to rootFolder ManagedObjectReference:Folder ha-folder-root (ha-folder-root)
4. Find childEntity ManagedObjectReference:ManagedEntity[] ha-datacenter (ha-datacenter)
5. Choose hostFolder ManagedObjectReference:Folder ha-folder-host (host)
6. Then find childEntity ManagedObjectReference:ManagedEntity[] ha-compute-res
7. Choose host ManagedObjectReference:HostSystem[] ha-host
8. Finally, go to runtime HostRuntimeInfo runtime
9. On doing that, navigate to healthSystemRuntime HealthSystemRuntime healthSystemRuntime
10. Click on systemHealthInfo HostSystemHealthInfo systemHealthInfo
11. Find numericSensorInfo HostNumericSensorInfo[] numericSensorInfo
12. Compare the value in the following string name string "VMware Rollup Health State"

Hope it helps! If you need any assistance with this, please contact our support team directly.
Jpurdy79
Influencer
Posts: 17
Liked: never
Joined: Jun 18, 2010 1:46 pm
Full Name: Jonathan Purdy
Location: Franklin, TN

Re: v5 false alarm "Host hardware sensor status changed"

Post by Jpurdy79 »

Thanks. I'm going to open a ticket with VMware support. What I'm seeing is that when viewing the "VMware Rollup Health State" string from the vCenter MOB, the key is "unknown" and the summary is "Cannot report on the current health state of the element". Whereas, if I veiw the "VMware Rollup Health State" string from the Host MOB, the key is "green" and the summary is "Sensor is operating under normal conditions".
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Jonathan, you're welcome. I would appreciate it if you could update this topic with VMware's official resolution.
Jpurdy79
Influencer
Posts: 17
Liked: never
Joined: Jun 18, 2010 1:46 pm
Full Name: Jonathan Purdy
Location: Franklin, TN

Re: v5 false alarm "Host hardware sensor status changed"

Post by Jpurdy79 »

Will do.
axs@moffatt.com
Novice
Posts: 9
Liked: 1 time
Joined: Oct 21, 2010 2:29 pm
Full Name: Andrew Seal
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by axs@moffatt.com » 1 person likes this post

A quick note on this thread. I had the same issue as described above and removing and rejoining the host did fix this issue but seems to have broken my replication jobs completely. What I have learned now is that if you reboot the host server without first putting it into MM it will show this fault. The solution for me was to put the server into MM, reboot, bring server out of MM and the "viola" no more error. I wish I would have known this before I removed and rejoined the host as it looks like I am going to need to rebuild and start over on all of my replication jobs...
aqualityplace
Influencer
Posts: 19
Liked: 1 time
Joined: Dec 11, 2009 8:58 am
Full Name: MC
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by aqualityplace » 1 person likes this post

We found simply putting the host into maitenance mode and then exit maitenance mode cleared the error
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Michael, thanks for finding time to update this topic.
RumataRus
Enthusiast
Posts: 78
Liked: 2 times
Joined: Jun 21, 2010 5:30 am
Full Name: Dmitry Prokudin

"Host hardware sensor status changed" warning

Post by RumataRus »

[Merged with existing discussion]

I often see alarm "Host hardware sensor status changed" warning though there is no errors/warnings in vCenter about hardware status.
Is it possible to reduce the sensitivity of host hardware sensors in Veeam Monitor?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Dmitry,

This alarm is being triggered by a certain condition defined in the alarm properties. You can use Reporting tab to adjust the triggering frequency. However, If I were you I would try to investigate why you have this status changed for the host.
RumataRus
Enthusiast
Posts: 78
Liked: 2 times
Joined: Jun 21, 2010 5:30 am
Full Name: Dmitry Prokudin

Re: v5 false alarm "Host hardware sensor status changed"

Post by RumataRus »

Vitaliy S. wrote:Dmitry,
However, If I were you I would try to investigate why you have this status changed for the host.
How? vCenter Alarms and Logs are clean.
BTW, if I clear ALL alarms in Virtual Infrastructure I receive "Host hardware sensor status changed" warning in about 10 seconds on two of my three hosts.
If I do it again, I receive the same again on the same hosts. :) It is stable.
RumataRus
Enthusiast
Posts: 78
Liked: 2 times
Joined: Jun 21, 2010 5:30 am
Full Name: Dmitry Prokudin

Re: v5 false alarm "Host hardware sensor status changed"

Post by RumataRus »

aqualityplace wrote:We found simply putting the host into maintenance mode and then exit maintenance mode cleared the error
Wow! It really helps!
Fiskepudding
Expert
Posts: 213
Liked: 26 times
Joined: Feb 01, 2012 7:24 am
Full Name: Espen Dykesteen
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Fiskepudding » 1 person likes this post

Great post. Altho it is old, I just want to chip in my exprience.

"VMware Rollup Health State" reported: "Cannot report on the current health state of the element"

This was when browsing the MOB on the vCetner server
When browsing directly on the HOST it reported: "Sensor is operating under normal conditions"

We took the HOST to manintanance mode and rebooted it. As suggested.
Took it out of maintanance mode, waited 5 minutes and looked in vCenter's MOB.
Still the same issue, "Cannot report on the current health state of the element".

Waited even longer, logged of MOB and back on. Nothing changed.

Finaly we rebooted the vCenter server, and all was good when it came back online.
So a tip migt be to also restart vCenter.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v5 false alarm "Host hardware sensor status changed"

Post by Vitaliy S. »

Thanks for sharing your resolution with the community.
Post Reply

Who is online

Users browsing this forum: No registered users and 16 guests