OpsMgr Warnings from the VEM

[maekee]

Hi everyone,
A couple of days ago we imported the nWorks MP and installed the VIC on a separate virtual VEM from the RMS.
Everything seems to work fine except some warning-messages that i cant explain.

The following messages repeats from about 5 to 30 times per hour.
They are all Warning-messages and looks like this:

Alert Rule: WMI Probe Module Execution Failure
Description: Object enumeration failed Query: 'select * from VMHOSTCLUSTER' HRESULT: 0x80041006 Details: Out of memory One or more workflows were affected by this. Workflow name: nworks.VMware.VEM.VMCCR.Collect.memoryUsedPct Instance name: c2 Instance ID: {94F0A9FA-8327-7B62-C4C4-D13585B10933} Management group: MOM_MG
3 repeats in 60 minutes

Same Rule, (WMI Probe Module Execution Failure)
Different Description:
Object enumeration failed Query: 'select * from VMHOSTCLUSTER' HRESULT: 0x80041006 Details: Out of memory One or more workflows were affected by this.
Workflow name: nworks.VMware.VEM.VMCCR.Collect.cpuUsedPct Instance name: c2 Instance ID: {94F0A9FA-8327-7B62-C4C4-D13585B10933} Management group: MOM_MG
3 repeats in 60 minutes

Same Rule, (WMI Probe Module Execution Failure)
Different Description:
Object enumeration failed Query: 'select * from VMHOSTSTATS' HRESULT: 0x80041006 Details: Out of memory One or more workflows were affected by this.
Workflow name: many Instance name: many Instance ID: many Management group: MOM_MG
11 repeats in 60 minutes

Same Rule, (WMI Probe Module Execution Failure)
Different Description:
Object enumeration failed Query: 'select * from VMGuestDiskProperties' HRESULT: 0x80041006 Details: Out of memory One or more workflows were affected by this.
Workflow name: nworks.VMware.VEM.VMGUESTVIRTUALDISK.Discovery Instance name: SERVERX Instance ID: {EEA943F1-1136-0825-9500-AFF3A8A21060} Management group: MOM_MG
27 repeats in 60 minutes

What is this?


/Maekee

[Alec King]

Hi Maekee

These errors indicate a problem within WMI itself....usually related to performance/resources.

Can you confirm what version of -
Windows
nworks MP
nworks Collector
...you are running?

What size is the VMware environment you are monitoring? (How many Hosts and VMs?)

Are you seeing any other error messages relating to WMI? (for example in the VEMs Application log)

Thanks!
Alec

[maekee]

Hi,
Its really anoying, get it about 300 in three days times in the Operations Manager Eventlog.
"Health Service Modules", EventID 10409

> Windows on the VEM: Windows Server 2003 Enterprise 64-bit
> nWorks MP: the latest (Version 3.6.1)
> nWorks Collector: 3.6.1 (Build 33375, UI Build: 25211)
> 6 VMWare Hosts, about 100 Virtual Guests.
> No WMI-errors in VEM App Event Log or System Log

/Maekee

Originally posted by Alec King

Hi Maekee

These errors indicate a problem within WMI itself....usually related to performance/resources.

Can you confirm what version of -
Windows
nworks MP
nworks Collector
...you are running?

What size is the VMware environment you are monitoring? (How many Hosts and VMs?)

Are you seeing any other error messages relating to WMI? (for example in the VEMs Application log)

Thanks!
Alec

[Alec King]

Hi Maekee,

Sorry for the delay in replying...I have been travelling.

There are some known issues with SCOM and WMI (I think you had an email from our support team on this)
Can you confirm you are running SCOM SP1?

Is this server dedicated to nworks, or sharing with another app? Is it a VM or physical server?

You can try clearing and rebuilding the wmi repository on the nworks server with cmd:
WMIADAP.exe /f

Let me know if that helps,

Thanks
Alec

[maekee]

Hi Alec,
Yes, i got a mail from your Nick that its ok to disable the Alerts because of a Microsoft bug somehow. So i disabled the script on these machine. Also get a few other errors and warnings that are repeating over and over on the VEM-mashine in the following order:

Event ID: 10409
Source Health Service Modules
Type: Warning
Computer VEM-Server
Description:
Object enumeration failed
Query: 'select * from VMGUESTSTATS'

HRESULT: 0x80004003
Details: Invalid pointer

One or more workflows were affected by this.

Workflow name: many
Instance name: many
Instance ID: many
Management group: MGXYZ

Followed by:

Event ID: 11904
Source Health Service Modules
Type: Error
Computer VEM-Server
Description:
The Microsoft Operations Manager Expression Filter
Module failed to query the delivered item, item was dropped.

Property Expression: Parameter 1

Error: 0x80004005

One or more workflows were affected by this.

Workflow name: MomUIGeneratedRule7b640e9b41fc40619a54e3373d89a776
Instance name: VEMSRV.domain.local
Instance ID: {1262B87D-8BF8-FBD8-D325-CDA36097A4EB}
Management group: MGXYZ

Followed by:
Event ID: 1103
Source HealthService
Type: Warning
Computer VEM-Server
Description:
Summary: 111 rule(s)/monitor(s) failed and got unloaded, 1 of them reached the failure limit that prevents automatic reload. Management group "MGXYZ". This is summary only event, please see other events with descriptions of unloaded rule(s)/monitor(s).


Why??
Help!

/Maekee

[Alec King]

Hi Maekee,

Well - the event 11904 is nothing to do with the nworks MP - it refers to "MomUIGeneratedRule7b640e9b41fc40619a54e3373d89a776".
This is a custom rule generated in the Ops Mgr Console Authoring section. For that issue I suspect that the property 'Parameter 1' is either formatted wrong, does not exist, or is the wrong data type.

For the first event, 10409 - that is an nworks DataSource, but again indicates a problem within WMI rather than the MP.
Were you able to try the WMIADAP /f command?

How much RAM does this machine have? How much CPU?
Is it a VM or physical box?
Are you seeing high cpu/memory % usage?

Thanks
Alec

[maekee]

Hi there,
Ok, i will find out what rule the event id 11904 referrs to.. thanks for the info

About 10409, yes i ran the wmiadap /f, no more errors have arrived since yesterday.
So i will wait and see if the problem was solved then and there.

If not, i'll get back in touch

About your Q's:
Its a VM with 2GB RAM and 3GHz CPU
No high CPU/Memory % Usage what i can see.. maybe peaks that i dont see

/Maekee

[gregman]

Hi, We have the same errors

The errors are in Operations Manager eventlog, with error event 10409.

Is there a fix?

Environment:
SCOM 2007 SP1, KB954049, KB951979, KB911380 (W2003 R2, SP2, x64)
MP 4.0.1.22
-------------
VICVEM (W2003 R2, SP2, x64)
Collector Service
Version 4.0.1
Build 4.0.1.1
UI Version 4.0.0
UI Build 4.0.0.48

1x Virtual Center
53x Virtual Machine

Object enumeration failed

Query: 'select * from SENSORSTATS'

HRESULT: 0x80004005
Details: Unspecified error


One or more workflows were affected by this.

Workflow name: nworks.VMware.VEM.VMHOST.VMSENSOR.TEMPERATURE.Discovery
Instance name: TEMP
Instance ID: {E56AB979-D6C1-8CCF-002E-4C8594E8413C}
Management group: SCOM_PROD01

For more information, see Help and Support Center at

------------------

Object enumeration failed

Query: 'select * from RESOURCEPOOL'

HRESULT: 0x80004005
Details: Unspecified error


One or more workflows were affected by this.

Workflow name: nworks.VMware.VEM.VMPOOL.PropertyDiscovery
Instance name: nWorks
Instance ID: {35633D27-77B5-8D5C-29FB-369CDEB35F32}
Management group: SCOM_PROD01

For more information, see Help and Support Center at

--------------------------

Object enumeration failed

Query: 'select * from VMDiskProperties'

HRESULT: 0x80004005
Details: Unspecified error


One or more workflows were affected by this.

Workflow name: nworks.VMware.VEM.VMGUESTVIRTUALDISK.Discovery
Instance name: vmi882as
Instance ID: {8DD50AE3-199C-A44E-ADF3-66F6B86A172B}
Management group: SCOM_PROD01

For more information, see Help and Support Center at

----------------------

Object enumeration failed

Query: 'select * from HARDWARESTATUS'

HRESULT: 0x80004005
Details: Unspecified error


One or more workflows were affected by this.

Workflow name: nworks.VMware.VEM.VMHOST.HARDWARESTATUS.Discovery
Instance name: Hardware
Instance ID: {6063B40E-E03B-0C53-EBFB-6C3A506A7579}
Management group: SCOM_PROD01

For more information, see Help and Support Center at