Since v6.5 upgrade "Host hardware sensor status changed"

Monitoring and reporting for Veeam Backup & Replication, VMware vSphere and Microsoft Hyper-V

Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby AlainRussell » Mon Nov 05, 2012 8:29 pm

I've seen this error on all our hosts since the 6.5 upgrade. Pre 6.5 this error could be cleared by entering maintenance mode and then exiting. Since v6.5 I cannot clear this error .. I've tried the maintenance mode fix, restarting the host, restarting Veeam one but nothing seems to clear the error.

Does anyone have any ideas?
Thanks
Alain
AlainRussell
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Sat Aug 27, 2011 12:04 am
Full Name: Alain Russell

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaliy S. » Tue Nov 06, 2012 7:43 am

Hi Alain,

Could you please tell me what sensor has changed its status? What does vSphere Client show for the same sensor?

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19557
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby AlainRussell » Wed Nov 07, 2012 7:34 am

Hi Vitaliy,

VCenter is not showing any errors currently (All Hardware sensors are showing as Normal), Veeam One is showing all the sensors for each host as "equal Unknown".

Thanks
Alain
AlainRussell
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Sat Aug 27, 2011 12:04 am
Full Name: Alain Russell

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaliy S. » Wed Nov 07, 2012 7:37 am

In v6.5 we have switched to a more advanced host hardware collection method (CIM over XML) and for some sensors VMware doesn't have any state and returns “unknown” state, which leads to triggered alarms. Please remove the rule that triggers "Host hardware sensor status changed" alarm when sensor changes its state to “Unknown”. Hope this helps!
Vitaliy S.
Veeam Software
 
Posts: 19557
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby AlainRussell » Wed Nov 07, 2012 6:54 pm

Thanks, that sorted it!
AlainRussell
Enthusiast
 
Posts: 47
Liked: 1 time
Joined: Sat Aug 27, 2011 12:04 am
Full Name: Alain Russell

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby ch@switch-it.se » Thu Nov 08, 2012 6:24 pm

Hi!

I have the same problem with HP ProLiant servers since the upgrade to 6.5.

Is it an ok (safe) solution to disable that rule? I have had issues earlier with servers that randomly reported all sensors with status unkown. I mean there is a risk that a faulty server reports status unknown. If all sensors are reporting unknown, then you don't know if server hardware are ok. It could be stuck in status unkown and you think everything is good.

/Christian
ch@switch-it.se
Novice
 
Posts: 4
Liked: never
Joined: Thu Oct 25, 2012 7:14 am
Full Name: Christian Hellemar

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaliy S. » Thu Nov 08, 2012 10:01 pm

Hi Christian,

Mostly you should be concerned about sensors which have either warning or error health state, but your point is also valid. Since "Unknown" status is retrieved for both, sensors without any state and sensors that have just switched to this state, I'm not sure there is way to differentiate these two conditions.

As a workaround you may want to leave this rule, but assign a different severity level (info) when host hardware sensor switches to "unknown" status.

Thank you!
Vitaliy S.
Veeam Software
 
Posts: 19557
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaly.Mzokov » Thu Nov 08, 2012 10:04 pm

In addition to Vitaliy's post...Prior to v6.5 we've seen that some hardware sensors didn't return the correct status due to a known VMware issue - when host's sensors display wrong status via vCenter Server's Managed Objects Browser (MOB). In v6.5 we are using more accurate hardware monitoring protocol CIM-XML which completely eliminates such behavior. Be aware that vSphere Client uses the same protocol to obtain sensors' states.

Moreover, since Veeam ONE v6.5 the hardware monitoring became more granular, and you might notice way more sensors available in v6.5 in comparison to v6.0. But, as Vitaliy S. has already mentioned, for some sensors VMware doesn't have any state and returns just “unknown” status. Please keep in mind that the correct state for "well-known sensors" is always available via CIM-XML (which is utilized by Veeam ONE by default).
If NOT, then you have to talk to VMware and get this issue addressed ASAP. :wink:

Following this logic, in most cases "unknown" state will be returned for sensors that VMware doesn't have any information about. Thus, answering your question, Christian - "Yes, I think this is a safe solution to disable the rule after upgrade"!

Hope this helps!
-VM.
Vitaly.Mzokov
Expert
 
Posts: 173
Liked: 25 times
Joined: Mon Aug 23, 2010 8:30 am
Full Name: Vitaliy Mzokov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby ch@switch-it.se » Fri Nov 09, 2012 12:09 am

Thank you guys for the info/comments :)

I will disable the rule.

/Christian
ch@switch-it.se
Novice
 
Posts: 4
Liked: never
Joined: Thu Oct 25, 2012 7:14 am
Full Name: Christian Hellemar

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby cffit » Mon Nov 19, 2012 10:29 pm

We also have current HP Proliant DL360 G6 and G7 Servers that showed fine until the 6.5 upgrade, and now they all show this error. This is pretty mainstream server hardware and it surprises me that the solution is to disable the monitor.

If I do call into VMWare support, what do I ask of them since their monitoring is showing up fine in their client?
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaliy S. » Tue Nov 20, 2012 7:50 am

Christensen,

If hardware sensor in vSphere Client has a "healthy" state while Monitor Client reports the opposite for the same sensor, then you need to contact our support team for further investigation. Your issue might not be related to the problem discussed in this thread.

Thanks!
Vitaliy S.
Veeam Software
 
Posts: 19557
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby cffit » Tue Nov 20, 2012 10:42 pm

I take that back, in VMware these sensors also show as unknown too.

Here's the issue though, I have the HP branded ESXi 5 install, and just to make sure, I installed the HP Offline bundle with all the newest CIM agents. The sensors still show as "Unknown" in VMWare. I had someone I know that has HP, Dell and IBM servers check theirs out. They say that on all three platforms they have the same sensors show up as "Unknown" too. And he has the HP branded ESXi 5 install on them.

With this said, if those three server brands have this issue, I can't imagine any server out there where all the sensors communicate and report properly. Is there any case where they do all work? And VEEAM 6.5 is happy and clear (without manually disabling this alarm)? Has anyone had any luck with this?
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby Vitaliy S. » Wed Nov 21, 2012 9:06 am

The hosts in my lab are not branded ones, so I don't have a rich set of hardware sensors to monitor, only few critical sensors are available and all of them communicate their status properly.
Vitaliy S.
Veeam Software
 
Posts: 19557
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby cffit » Wed Nov 21, 2012 2:09 pm

Christian Hellemar, did you have any luck with your HP Proliant servers in getting them to work? Or did you just remove the rule that monitors them?
cffit
Expert
 
Posts: 338
Liked: 33 times
Joined: Fri Jan 20, 2012 2:36 pm
Full Name: Christensen Farms

Re: Since v6.5 upgrade "Host hardware sensor status changed"

Veeam Logoby ijefferson » Mon Nov 26, 2012 3:06 pm

Hi

I'm having the same issue since upgrading, on a Dell T410 the following sensors are reporting as unknown:

Code: Select all
Sensor "Disk Drive Bay 1 Cable SAS A 0: Connected - Assert" equal Unknown
Sensor "Disk Drive Bay 1 Cable SAS B 0: Connected - Assert" equal Unknown
Sensor "System Board 1 Status 0: Connected - Assert" equal Unknown
Sensor "DIMM_A1" equal Unknown Sensor "DIMM_A2" equal Unknown
Sensor "DIMM_A3" equal Unknown Sensor "DIMM_A4" equal Unknown
Sensor "DIMM_B1" equal Unknown Sensor "DIMM_B2" equal Unknown
Sensor "DIMM_B3" equal Unknown Sensor "DIMM_B4" equal Unknown
Sensor "vmnic0" equal Unknown Sensor "vmnic1" equal Unknown
Sensor "vmnic2" equal Unknown Sensor "vmnic3" equal Unknown
Sensor "System Board 1 Power Optimized 0: OEM - Assert" equal Unknown
Sensor "System Board 1 Power Optimized 0: Unknown - Deassert" equal Unknown
Sensor "Disk Drive Bay 1 Presence 0: Absent - Deassert" equal Unknown
Sensor "Disk Drive Bay 1 Presence 0: Present - Assert" equal Unknown
Sensor "Power Supply 1 Presence 0: Absent - Deassert" equal Unknown
Sensor "Power Supply 1 Presence 0: Present - Assert" equal Unknown
Sensor "Power Supply 2 Presence 0: Absent - Deassert" equal Unknown
Sensor "Power Supply 2 Presence 0: Present - Assert" equal Unknown
Sensor "Processor 1 Presence 0: Absent - Deassert" equal Unknown
Sensor "Processor 1 Presence 0: Present - Assert" equal Unknown
Sensor "Processor 2 Presence 0: Absent - Deassert" equal Unknown
Sensor "Processor 2 Presence 0: Present - Assert" equal Unknown
Sensor "System Board 1 HEATSINK PRES 0: Absent - Deassert" equal Unknown
Sensor "System Board 1 HEATSINK PRES 0: Present - Assert" equal Unknown
Sensor "System Board 1 iDRAC6 Ent PRES 0: Absent - Assert" equal Unknown
Sensor "System Board 1 iDRAC6 Ent PRES 0: Present - Deassert" equal Unknown
Sensor "CPU1 Level-1 Cache DeviceID:1041.01" equal Unknown
Sensor "CPU1 Level-2 Cache DeviceID:1042.01" equal Unknown
Sensor "CPU1 Level-3 Cache DeviceID:1043.01" equal Unknown
Sensor "CPU2 Level-1 Cache DeviceID:1041.02" equal Unknown
Sensor "CPU2 Level-2 Cache DeviceID:1042.02" equal Unknown
Sensor "CPU2 Level-3 C
ijefferson
Influencer
 
Posts: 14
Liked: 1 time
Joined: Wed Nov 21, 2012 10:53 am
Full Name: Ian Jefferson

Next

Return to Veeam ONE



Who is online

Users browsing this forum: No registered users and 4 guests