Linux Agent Entire Computer Backup, Instant Recovery to vSphere Boot Issue

[rleon]

Hi all,

I encountered the following issue in my testing environment. Currently holding off deployment for a customer until a solution is found.
Veeam Server version is 11.0.0.837 P20210324, running on Win2019, with Repo on ReFS.
Veeam Linux Agent veeam-5.0.0.4318-1 for CentOS 8.3.2011.
CentOS Agent is a VM on vSphere 7.0U1. (Using a VM for testing only. This will be a real physical rack CentOS server in a production environment)
Agent job backup "Entire Computer" completed successfully to the Veeam Server Repo.

I'm trying to test out the "Instant Recovery" to vSphere feature for Linux, but encountered the following problems.
The Instant Recovery job actually completed successfully too, and allowed for migration/vMotion to the real datastore.
The problem stayed the same whether you migrate or not though.
Please refer to the following screenshots.

Screenshot1: The Instant Recovery VM looks weird, and boots only into dracut emergency mode.
https://imgur.com/1fbEWdF

Screenshot2: Tried following this KB, but doesn't seem to be the same issue: https://www.veeam.com/kb2669
https://imgur.com/gYJ3Kqq

Strangely, the Linux Recovery Media (generic ISO) method worked without issues on a new VM.
No weird extra disk, and boots normally back to CentOS.
I guess this Recovery Media method is already a workaround for this problem. But still I would prefer to find out why Instant Recovery doesn't work...
Any ideas?
Thanks in advance!

[rleon]

Screenshot3: For some reason the Instant Recovery VM used up over 10x more disk space than the source VM
https://imgur.com/N8aMGOC

[PTide]

Hi,

I believe that I've already seen a similar bug in our bug-tracker. In order to connect the dots, I would like to ask you to open a support case and post your case ID so that I could pass it to QC.

Thanks!

[rleon]

Hi PTide,

Case #04771809
Didn't know I could create a case for product 'evaluation'.
Thanks.

[PTide]

Perfect. The case has been queued for escalation.

Thanks!

[rleon]

Screenshot4: https://imgur.com/awNWi2F
The support asked for this information, which I sent him as a screenshot.
Thought I'd post it back here too to document the progress.

[rleon]

Screenshot5: https://imgur.com/N2LdZ7J
Veeam support replied that this is a known issue for RHEL/CentOS 8.3 Instant Recovery to Hyper-V and Azure. But they need to investigate further for VMware environments.
I thought I'd go ahead and test IR against CentOS 7.9. The results are shown in the image.
Summary: At least it boots now, and I can login as root. But there are some other issues...

[rleon]

In case someone may find this useful, long story short, When Linux-Agent backups get IR'd as a VM:

1. Veeam selects e1000 (1GbE) for the NICs instead of VMXNET3(10GbE). This is by design for compatibility reasons.
2. The Linux OS detects NIC and MAC change and will change the NIC Device Naming. This means you can't expect the IR VM's networking to just start working and connecting, without having to manually change some settings using NMCLI or ifcfg files.
3. Veeam makes the IR'd VM virtual disks Thick Provisioned. This is by design by Veeam.
4. If the Linux has LVMs, first, the disk device itself gets a thick VMDK (Eg: sda gets a VMDK), then, each LVM Volume Group inside the disk device gets a thick VMDK (Eg: sda3 has a LVM VG, so this VG gets a VMDK). This explains why a 100GB source disk gets two thick IR VMDKs: 100GB VMDK for the source disk itself, another 98GB VMDK for the LVM VG. (Funny how the 98GB is just part of the 100GB in the original source.) This behavior is by design by Veeam
5. For Red Hat / CentOS 8.3, the https://www.veeam.com/kb2669 workaround does work. You just need to follow the Debian steps if the RH/CentOS steps don't work.

That about sums it up. Special thanks to Phillip Yang from Veeam Support.