VeeamONE Service monitoring bug/feature request

[ChrisGundry]

Hi,

I was VERY pleased to see that Veeam have now added a function we have wanted for a long time, Windows Guest service monitoring! We can now set alerts for Windows VMs.

However I was disappointed to find that unfortunately we can't use it to replace our current utility which is a real shame. The reasons are:
1. Alert delay - When a Windows service stops VeeamONE doesn't alert quickly enough. I have spoken to support on case # 03518044 and they say that in their testing from service stopping to Veeam alerting is around 2-3 minutes. This is too long for us and our current solution alerts in 20-30 seconds. They advised there was a registry option to change the GuestServicesUpdatePeriodInSec lower than the default of 20 seconds. However this didn't seem to change anything when set to 15 seconds as the test results were basically the same, it might have been 15 seconds faster, but not noticeable really. Example:
Windows event log:
4/22/2019 2:26:49 PM The Print Spooler service entered the stopped state.

In Veeam ONE logs:
22.04.19 14:29:21 Inform 0x17F4 Alarms "UPDATE Alarm(1799): Name = Spooler on VBRu3(alarm_id = 350, assign_id = 357), Status = 4, Entity = 2388, Message = Spooler service state is Stopped"

Just under 2 minutes, not bad but if the default check is every 20 seconds, why does it take 2-3 minutes to alert? Support don't seem to 'get it' and can't seem to tell me why...

But in our environment this is more like 3-4 minutes, which is much too long when our current solution does this in 20-30 seconds!

2. Adding multiple similar services for a single server (an alert scoped to a single server) is laborious. For example lets say we want to add all the SQL related services on server x. We have to go through a several step wizard to do that. You can't just copy the rule condition and change the service name. Or better yet create a rule of some kind that says SQL* or similar. We have a SQL server that runs several instances, which means multiple SQLSERVER$xxxx instances where xxxxx is the name of the instance. This also means there are several other services all with a similar naming convention like SQL Server Agent being SQLSERVERAGENT$xxxxx. There are maybe 6 services like this, if we have 5 instances then that is 30 services to monitor. Adding each of those you have to go through a wizard and it takes forever. It's just not very practical for adding multiple services to a rule.

We can work around both of these things with simple PowerShell scripts/windows scheduled tasks which are much easier to configure and run, but we don't get the alerts showing in the Veeam dashboard, which is a shame.

If Veeam can see it through and fix these 2 issues then it could be a really nice solution.

Thanks!

[Shestakov]

Hi Chris,
1. Let me talk to developers and QA. I'll write back once I have more information.
2. You can create alarms from the Infrastructure View pane. Click on the desired VM in the tree, then go to Services tab. There you can select with CTRL+Click all services you want to be alarmed against and click "Create Alarm..." button. Alarm configuration window will appear. There you can go to the "Assignment" tab. You will see 1 VM in the scope. Click "Add..." to assign the same alarm settings to 4 more VMs.
That will save lots of time.
Anyways, I like your idea to add "Clone" option to the alarm rules, it totally makes sense. I'll add it to the product roadmap.

[ChrisGundry]

Thank you Nikita, that will make it a million times easier, I don't know how I didn't realise that was an option! That would make our initial setup a LOT easier! Adding additional services later on would still be a bit of a pain until a 'clone' option was added, but we could work through that I think. This initial setup was the biggest hurdle, migrating a LOT of services over to VeeamONE to be monitored would have taken forever.

If the timing can be improved that would hopefully enable us to use VeeamONE and have this all in one place!

Thanks again

[Shestakov]

Thanks for the kind words, Chris!
By the way, what tools are you currently using for the services monitoring?

[Shestakov]

1. For now, 2-3 minutes is indeed the expected trigger time. We will try to speed it up to 20-30 seconds in the next updates.
Thanks for the feedback and the feature request!

[ChrisGundry]

Hi Nikita. For some reason I can't see your last post on the topic, unless I click reply, very odd.

Can you elaborate on why though? The cycle is supposed to be every 20 or 60 seconds, so why does it take 2-4 minutes to trigger an alarm?

Thanks

[Shestakov]

That's how the current data collector works. Default interval is 1 minute. We have an idea how to speed it up in current release, but need to test it first.

[Shestakov]

Unfortunately the fix doesn't work properly, so we will work on in the next versions.
Thanks!

[Vitaliy S.]

Chris, good news! Our R&D team didn't stop working on this and has managed to overcome all the problems when fixing this issue, so please contact our support team to obtain a hotfix (currently in the final stage of QC process) that will decrease the time required to detect problems with services and processes (Fix ID 170388).

[ChrisGundry]

Hi Vitaliy

That is great news, thank you very much.I have logged a case #03569402 for the hotfix. Looking forward to testing/using it!

Thanks

Chris

[Vitaliy S.]

Please let us know the updated results after applying the hotfix.

[ChrisGundry]

Hi Vitaliy

Still waiting on the hotfix from support to test this...

Thanks

Chris

[Shestakov]

Hi Chris,
Let me double-check the case status with support management.
I'll let you know the result.
Thanks

[ChrisGundry]

Hi Nikita

I have just received a hotfix to test from the QA/support team and I can confirm it is working really well. In all my tests so far it is now detecting in 30-32 seconds which is a big improvement and is much more usable for us. We can now look at moving our service monitoring into VeeamONE as well, which is great for us!

Many thanks for your help on this.

Chris

[Shestakov]

Thanks for the update, Chris!
I'll forward your kudos to R&D team.