Collector event queue has reached it's limit

Unleash the power of System Center for vSphere and Hyper-V | Veeam Task Manager for Hyper-V

Collector event queue has reached it's limit

Veeam Logoby mwinchester » Fri Apr 01, 2011 10:24 am

Hi All

We got this alert today and am looking for some advice on calculating a good figure to increase the limit too, also what if any performance implications are there on the collector ?

Alert: nworks Collector: VC Event Queue has reached its limit
Source: camvc001
Path: vcamsccs001.EUROPE.PPDI.LOCAL
Last modified by: System
Last modified time: 01/04/2011 04:40:52
Alert description: VC220 The internal events queue for the nworks Collector has reached its limit. vCenter camvc001 has generated more than the current Collector capacity of 75 events per 30 seconds, therefore some events could be lost. Check for an event storm in vCenter.


This event generated by the nworks collector For more information see http://veeam.com/support/events.aspx?EvtID=119


Alert view link: "http://CAMRMS01:51908/default.aspx?DisplayMode=Pivot&AlertID=%7b3abb4018-08f2-4142-80c9-6f6e5fbf8a8d%7d"
Notification subscription ID generating this message: {E0043DE2-681D-B3BE-D2B8-F9C747ACED6E}


Regards

Mark
mwinchester
Novice
 
Posts: 3
Liked: never
Joined: Thu Apr 29, 2010 10:41 am
Full Name: Mark Winchester

Re: Collector event queue has reached it's limit

Veeam Logoby vBPav » Wed Apr 06, 2011 10:59 pm

Mark,

You can increase this up to 1000 events / 30 seconds I believe. This will obviously increase the amount of data the nworks Collector will have to query thus increasing CPU utilization. As long as you monitor the usual metrics on each nworks Collector you should be ok.

VM\cpuUsed
OS\Processor Queue
Agent\SendQueueLength

FYI - this is a new feature implemented in nworks MP 5.6, which is why you may have "just" started to see the behavior.

Cheers!

Brian Pavnick - Solutions Architect
Brian Pavnick | Cireson| Solutions Architect

- Follow me on Twitter @ vbpav
- Reach me on e-mail @ brian.pavnick@cireson.com
vBPav
Expert
 
Posts: 181
Liked: 13 times
Joined: Wed Jan 13, 2010 6:08 pm
Full Name: Brian Pavnick

Re: Collector event queue has reached it's limit

Veeam Logoby mike.bosco@dcsg.com » Mon Jun 13, 2011 2:45 pm

I started recieving this alert after upgrading to 5.6. The default is 75 events per 30 seconds, how can I create an override to change this?
mike.bosco@dcsg.com
Lurker
 
Posts: 2
Liked: never
Joined: Wed Feb 09, 2011 5:03 pm
Full Name: Mike Bosco

Re: Collector event queue has reached it's limit

Veeam Logoby Christopher Keyaert » Thu Jun 30, 2011 1:56 pm

mike.bosco@dcsg.com wrote:I started recieving this alert after upgrading to 5.6. The default is 75 events per 30 seconds, how can I create an override to change this?

I've got the same alert time to time in version 5.6, so I'm also interested to know how to increase this value.

Thanks
Christopher
Christopher Keyaert
Enthusiast
 
Posts: 30
Liked: never
Joined: Thu Jul 15, 2010 9:33 am
Full Name: Christopher Keyaert

Re: Collector event queue has reached it's limit

Veeam Logoby aladd » Wed Jul 06, 2011 3:14 pm

Any chance we can get an answer to this question? I have a very busy VC and tweaking this queue would save me a lot of useless messages.

I could open a support case but since this appears to be a common issue having the solution in a forum would help someone else down the line.

Thanks,

Allen
aladd
Novice
 
Posts: 4
Liked: never
Joined: Tue Apr 06, 2010 1:43 pm
Full Name: Allen Ladd

Re: Collector event queue has reached it's limit

Veeam Logoby Alec King » Wed Jul 06, 2011 4:09 pm 1 person likes this post

Greetings all,

This new setting (since 5.6) is available in the nworks Enterprise Manager UI. (You cannot use an Ops Mgr override to change this)

Setting is called Event Queue Limit, and you can find it in Advanced Settings in nworks Enterprise Manager, on Enterprise Manager tab when the top node Enterprise Manager is selected.

I would recommend only increasing to around 100 - 150 or so, to see if this stops the alerts. Any increase in the event page size we request from vCenter could have a performance impact on VC.
Note the absolute maximum setting is 1000.

Please note - there are a couple of 'false positives' for this alert (which we will fix in the next update). So do not assume that you need to increase this setting. You can get a false alert if -
(1) the Collector service was stopped for some time. Then the Collector sees the gap in events while it was stopped as 'missed events' and raises this alert.
(2) there is a failover of a monitoring job to an inactive Collector, and this Collector previously monitored the same VC at some time in the past. The Collector sees this gap in monitoring also as a 'gap' in events and raises the alert.
Please see the KB article on the monitor for full explanation of the monitor operation. (Also note, to have the full KB, you should have the latest MP, version 5.6.0.786)

Cheers,
Alec
Alec King
Vice President, Product Management
Veeam Software
Alec King
Veeam Software
 
Posts: 700
Liked: 116 times
Joined: Sun Jan 01, 2006 1:01 am

Re: Collector event queue has reached it's limit

Veeam Logoby aladd » Fri Jul 15, 2011 2:25 pm

Thanks Alec, That's exactly what I was looking for. :)
aladd
Novice
 
Posts: 4
Liked: never
Joined: Tue Apr 06, 2010 1:43 pm
Full Name: Allen Ladd

Re: Collector event queue has reached it's limit

Veeam Logoby aperson » Tue Oct 04, 2011 7:04 pm

How do we edit this default value?

Enterprise Manager version:5.6.0.696
aperson
Influencer
 
Posts: 14
Liked: never
Joined: Tue Oct 04, 2011 6:59 pm
Full Name: A Patel

Re: Collector event queue has reached it's limit

Veeam Logoby steve_mc » Thu Oct 06, 2011 2:45 pm 3 people like this post

Hello aperson,

We had similar issues with the VC events limit. (basically due to our busy vCenters)

To increase the limit access the Enterprise Manager, then open the Enterprise Manager tab. Select Enterprise Manager on the left hand column then open the Advanced Settings under Global Collection Settings section. Double click on the Event Queue Limit and set it to your desired value (ours is currently sitting at 350 with no issues). Save your settings. Access your collector servers and restart the nworks Virtual Infrastructure Collectors. Done

I would recommend putting your Veeam infrastructure into maintenance mode whilst you do.
steve_mc
Influencer
 
Posts: 12
Liked: 3 times
Joined: Tue Mar 01, 2011 3:11 pm

Re: Collector event queue has reached it's limit

Veeam Logoby aperson » Thu Oct 06, 2011 3:00 pm

Fantastic! Thanks for the "How to Do" !!
aperson
Influencer
 
Posts: 14
Liked: never
Joined: Tue Oct 04, 2011 6:59 pm
Full Name: A Patel

Re: Collector event queue has reached it's limit

Veeam Logoby felyjos » Fri Jun 15, 2012 5:38 pm

thnaks this saved me me
felyjos
Expert
 
Posts: 285
Liked: 3 times
Joined: Wed Jun 08, 2011 5:46 pm
Full Name: Dominique DUCHEMIN


Return to Veeam Management Pack for Microsoft System Center



Who is online

Users browsing this forum: No registered users and 4 guests