How are you dealing with consolidation power offs?

[electricd7]

Hello,

We are running Veeam in about a dozen customer sites and are slowly upgrading all sites to version 8 and ESXi version 5.5. We are starting to get stung more frequently with VMs powering off as a result a failed VMware snapshot consolidation. I realize that this isn't technically a Veeam problem, but Veeam exposes it quite often in the way it works. My question is, how are you all dealing with this problem? It wouldn't be so bad if the machines we automatically get powered back on, but thats not the case and it generates support calls at all hours of the night because of systems going down. I was thinking there would be a way to run a powershell script against the VMware environment to look for machines which are powered off which also have a specific VMware event in the last X minutes and then just power them back on, but thus far I haven't been successful in finding solid logic to use in the script. Any ideas?

Chad

[Shestakov]

Hello Chad,

Do you also use Veeam One?
If yes, it can be easily achieved by means of VM Power Status alarm. You can preset an action to send an email notification and/or send SNMP trap and/or run a script.

Veeam One reporter also contains a Powered Off VMs report which shows a list of all VMs that were in Powered Off state for a defined period of time. Thanks.

[electricd7]

We are not using Veeam One in most of the environments. Is the powered off alarm configured so that it does nothing if the power-off command is issued by a user and not an event trigged by a system action? I wouldn't want to be alarmed if a user powers down a machine as they are entitled to do so and have good reason to do so in most cases.

[Shestakov]

The alarm reports when VM changes its condition from On to Off. You can suppress it when certain task is performed and/or for a custom time period. You can assign the alarm to specific VMs as well. So if you know which VMs experience the problem, it`s feasible to assign the alarm to run a script to turn them on automatically.

There is also VM Uptime report which shows VMs uptime including VMs with lowest and highest uptime values. We are currently working on its enhancement including the information about reasons of VM power-off/reboot.
By the way, do you know what causes those VMs` deactivation?

[Vitaliy S.]

I would also try to find something in common between VMs that has these failures. For example, if all of them are located on datastores with latency issues, then I would suggest moving these VMs to other storage.