impact of changing CollectionInterval setting

[nico.weytens]

2 weeks ago we've upgraded to v6. The guys who originally designed and had set up the older infrastructure are no longer with us, so I'm pretty much on my own in figuring out why some settings were set, and if they still make sense for us. One thing where I doubt is the CollectionInterval setting for the collectors.

In the user guide I find:
Name CollectionInterval
Default Value 5 minutes
Description Defines time interval between two successive data collections from VMware servers. If the value is changed, it is necessary to restart all Collectors, to which the value applies.

For some reason this used to be set to 10 minutes in our production environment, while in our smaller environments it's simply the default.
Does doubling the value mean we will get half the data (thus stressing the vCenter/OpsMgr servers less), or does the amount of data that's pulled from vCenter to OpsMgr remain the same and is it simply a matter of how much we transfer at a certain interval?
I would imagine the value was changed from 5 to 10 mins because my predecessors saw a high load on vCenter and/or OpsMgr, so I should keep it like that, but I'm totally not sure of this.

Could someone elaborate on this, eg when would it be a good idea to change the default value?

[sergey.g]

Hi Nico,

It's hard to guess what was the purpose of setting collection interval to 10 minutes, so I'll just try to explain what this means for our product.

5 min Collection interval
Collector connects to vSphere host (vCenter or ESX) every 5 minutes and reads latest value
Collector publishes latest datapoint to WMI every 5 minutes
Collector instructs SCOM agent to collect new data every 5 minutes
SCOM saves data to the database every 5 minutes

10 min Collection interval
Collector connects to vSphere host (vCenter or ESX) every 10 minutes and reads two latest(5min interval) values
Collector rolls-up two 5min values to a single average and publishes data to WMI every 10 minutes
Collector instructs SCOM agent to collect new data every 10 minutes
SCOM saves data to the database every 10 minutes

So, with 10 min interval you get less datapoints in SCOM and it's an average of what we've received from vCenter, so you are not getting half the data, it's just "low-resolution" data instead of more detailed 5min stats.

In terms of vCenter less often queries could be more reliable in case of issues with vCenter performance. Also this change could be caused by some issues with SCOM infrastructure - either bad performance with collecting data by SCOM agents(which is less likely) or too big footprint of data in the database.

In v6 we've improved performance a lot, it should be much easier for SCOM agent to absorb data collected by our MP.
Database footprint for v6 should be smaller than for v5.7, so it may not be necessary to increase collection interval if the issue was caused by a database growth.
With respect to getting data from vCenter - there is almost no difference compared to 5.7, so it may be still necessary to keep collection interval at 10 minutes.

Let me know if you have any other questions.

Thanks.

[nico.weytens]

Thanks for the quick reply.
It might indeed be the DB growth that was the problem. If I look at the disk usage report of our OpsDB, it's currently at 52GB and the tables taking up most space are PerformanceData_xx. If I recall well having an OpsDB under 50GB is recommended.
Reverting back to the default collection interval would result in bigger PerformanceData tables, right? In that case I'd keep it on 10 mins.

[sergey.g]

Hi Nico,

I do not recall any recommendations for operational database. At the end - it's just a database and it can grow to very large size. But of course it can be limited by some other settings in your environment, so if it was necessary to keep amount of data below certain point, then increasing collection interval would help in reducing amount of data SCOM saves to the database.

Changing collection interval back from 10 to 5 minutes will indeed increase amount of data in Operational database. However it won't double it - minute by minute data is being stored only in PerformanceData_Raw tables, then SCOM rolls everything up to hourly averages and daily averages - in these tables amount of data will be the same. Also keep in mind that you have other Management Packs which can save data to the database. There is a report in the "System Center Core Monitoring Reports" library - "Data Volume by Management Pack", you can use it to estimate whether our MP is the main "contributor" for the database growth.

I hope this can help you with your investigations.

Let us know if you have any other questions or issues.

Thanks.

[nico.weytens]

ok, thanks Sergey.
I'll have a look at those reports and will decide on the interval based on that last piece of info