Customer deleted a few old replicas from the Veeam console - Replicas, ready. It turns out those replicas were actually live running VMs.
In hyper-v logs is clear that veeam did turn off machine and delete it. (also from export logs for specific hyper-v server).
I'm wondering how is that possible? Deleting running machines, isn't there some kind of safety builtin for such scenario?
Thanks for info.
-
- Service Provider
- Posts: 252
- Liked: 20 times
- Joined: Aug 02, 2011 9:30 pm
- Full Name: Matjaž Antloga
- Location: Celje, Slovenia
- Contact:
-
- Product Manager
- Posts: 2569
- Liked: 701 times
- Joined: Jun 14, 2013 9:30 am
- Full Name: Egor Yakovlev
- Location: Prague, Czech Republic
- Contact:
Re: VBR deleted running VMs on Hyper-V
Hi Matjaz
Strange case and for sure there are mechanics to avoid such thing to happen. Veeam will never delete running machines unless explicitly controlled to do so(with clear statements, exact VM names and confirmation dialogs).
Taking your quote "Replica, ready" = machine on the target side should have been shut down, that is definition of "Ready to start Failover" state, aka we are ready to power it on, if need be.
My guess would be that someone could have launched replica machines outside of Veeam boundaries, thus leaving product unaware of activities on said machine, did some work on those machines, and then instructed Veeam to delete what we have created in the first place...
It will be best to carefully troubleshoot the case with support - our engineers can reproduce actions taken step by step, and reveal the root of the problem.
/Thanks!
Strange case and for sure there are mechanics to avoid such thing to happen. Veeam will never delete running machines unless explicitly controlled to do so(with clear statements, exact VM names and confirmation dialogs).
Taking your quote "Replica, ready" = machine on the target side should have been shut down, that is definition of "Ready to start Failover" state, aka we are ready to power it on, if need be.
My guess would be that someone could have launched replica machines outside of Veeam boundaries, thus leaving product unaware of activities on said machine, did some work on those machines, and then instructed Veeam to delete what we have created in the first place...
It will be best to carefully troubleshoot the case with support - our engineers can reproduce actions taken step by step, and reveal the root of the problem.
/Thanks!
-
- Veteran
- Posts: 643
- Liked: 312 times
- Joined: Aug 04, 2019 2:57 pm
- Full Name: Harvey
- Contact:
Re: VBR deleted running VMs on Hyper-V
Hey Matjaž,
I would second what Egor says and make a Support Case, but I can almost guarantee you based on past experience with clients that this is because someone enabled the replica outside of Veeam, forgot they did this, then deleted the replica (from disk) within Veeam thinking they were just getting rid of an outdated machine.
The software cannot know that you've "manually failed over" to a machine, so if this happened, the "Replica" deleted from Veeam (from disk) was actually the production machine. If there wasn't a warning though that the Replica Delete From Disk was about to delete a running machine, there really should be. I think it's a fair statement to make that a Replica should never be running unexpectedly from Veeam's perspective, and if this is the case, Veeam should warn/wait instead of just happily deleting.
I would second what Egor says and make a Support Case, but I can almost guarantee you based on past experience with clients that this is because someone enabled the replica outside of Veeam, forgot they did this, then deleted the replica (from disk) within Veeam thinking they were just getting rid of an outdated machine.
The software cannot know that you've "manually failed over" to a machine, so if this happened, the "Replica" deleted from Veeam (from disk) was actually the production machine. If there wasn't a warning though that the Replica Delete From Disk was about to delete a running machine, there really should be. I think it's a fair statement to make that a Replica should never be running unexpectedly from Veeam's perspective, and if this is the case, Veeam should warn/wait instead of just happily deleting.
Who is online
Users browsing this forum: No registered users and 7 guests