Volume restore but hyperv vms are broken

[daystrom]

Case #04635124

A drive that hosted two important hyperv vms failed. I replaced the drive, and restored that volume (e:) from agent from the previous night's backup. It seemed to go well. The two vms were listed as OFF, but they would not start, with a message like:

An error occurred while attempting to start the selected virtual machine 'server' failed to change state
File not found

The restored drive held only the hyperv machines. I'm not sure what would be missing since the OS was not involved (windows 2012 r2). Because I needed these vms to be ready in the AM, I then turned to veeam backup and replication which I use to backup the vms. After first deleting the vm for the dc (that seemed to be required) I restore it, apparently successfully. But the windows 2012 r2 OS will not start, it offers troubleshooting etc. I've tried various fixes, the usual repair boot settings, and no change.

I'm posting this to the vbr forum because of the two I was fully confident that it would be able to deal with the vm itelf. But I don't understand why either of them failed. I don't see anything special about this restore scenario. I've never had issues like this before with other restores.

Using agent 4.0.1.2169
Using vbr 10.0.1.4854

[Vitaliy S.]

Just to make sure I understand everything > you were backing up VMs by using Veeam Agent (installed on the Hyper-V host) with a volume backup jobs, right? Please be aware that this is not recommended (and I guess this is the reason why VMs don't boot), instead, you need to use Veeam B&R to protect Hyper-V VMs using the native Hyper-V APIs.

To resolve your case, you can try creating a new VM and then attaching virtual disks from the restored VMs (might help).

[daystrom]

No that's not the scenario.
I backup the host, entire machine, with agent. Since an entire volume had to be restored I chose the volume restore option. That volume was the hard drive that had the hyper-v machines on it. I expected the volume restore to result in a fully restored copy of the original hdd, the E drive.
However both vms were not startable.
I then tried to restore the vm backups I had from vbr. The app server did restore without issue, as it turns out, but I didn't know when I made the first post. The first vm was the domain controller and after I wrote the first post here I found it was showing error 0xc00002e2. That's an error that only appears for domain controllers and indicates that active directory is broken in some manner. Since then I've restored the dc.
That leaves me with two questions.
[1] was it unreasonable to expect the Agent volume restore of E from good backups made the nigh before to result in bootable vms?
[2] why did the vbr restore restore of a good image result in a broken AD? This is totally unexpected.
It's the worst scare I've had with veeam products. They've been very reliable and easy for a non-expert like myself to work with.

[Vitaliy S.]

[1] was it unreasonable to expect the Agent volume restore of E from good backups made the nigh before to result in bootable vms?

Yes, VMs have to be backed up with Veeam B&R and not with Veeam Agent volume backup job.

[2] why did the vbr restore restore of a good image result in a broken AD? This is totally unexpected.
It's the worst scare I've had with veeam products. They've been very reliable and easy for a non-expert like myself to work with.

To make sure the DC VM can be restored successfully, you need to run your jobs with AAIP (application-aware processing) enabled and after the restore follow the steps mentioned in this KB > https://www.veeam.com/kb2119

[daystrom]

I will note the kb you linked to for dc vms restores in the future.

But you still don't understand the issue with Agent. All of the hyperv vms where hosted on the host server's E drive. The E drive was replaced in it's entirety. The vms were both dead on arrival. Why? I expected that Agent would restore the entire contents of the E drive, regardless of what they were, just like a disk clone operation would have, except that it would have been a copy from the day before. I cannot understand why Agent either can't handle this scenario properly, or why it failed in this particular case. Neither vm would start up. It's not the same issue as the dc starting and then failing, which was what the kb you reference deals with.

[Vitaliy S.]

The answer is pretty simple, VMs are not regular files and they have to be treated correctly when doing backups (due to the running state, VSS, snapshots etc). And this is possible when using Veeam B&R and native Hyper-V APIs.

[daystrom]

I've not had issues with restoring entire servers that include vms, and dcs. Is the difference that something in the non-restored sections of a host server cannot adapt itself to a full replacement of the drive that hosts the hyper-v assets? Would you expect the same issue to arise if the E drive physical disk was cloned, and the clone then replaced the original E drive?

[Vitaliy S.]

Well, if VMs were shut down and not producing any activity, then the method you're using for backing them up could work. But as I've referenced in my first post this is not recommended way of protecting your VMs, so issues might be expected.