Host-based backup of VMs running on Red Hat Virtualization, Oracle Linux Virtualization Manager, Scale Computing Hypercore, XCP-ng, HPE VM Essentials.
Post Reply
JaySt
Service Provider
Posts: 549
Liked: 126 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

[HPE VME] VirtIO driver injection - version used

Post by JaySt »

we got some feedback after trying the restore of a multi TB SQL database VM to HPE VME, now that we have the VirtIO driver injection with v13.1

injection seems to use storage driver 100.102.104.29000
Injected during restore of a Windows Server 2019 server.

However, we immediately saw database corruptions, storage io issues.
Seems that there is a known storage IO issue with the latest VirtIO drivers : https://github.com/virtio-win/kvm-guest ... ssues/1453
Reverting the driver to what's included in VirtIO 0.1.271 (the version just before the latest 0.1.285) seems to have fixed the storage IO error.

the issues seems to be fixed on github, but not yet merged into the latest stable. I think Veeam uses a version still affected by the issue.

any thoughts?
Veeam Certified Engineer
cody.ault
Veeam Software
Posts: 179
Liked: 89 times
Joined: Nov 04, 2010 2:53 pm
Full Name: Cody Ault
Contact:

Re: [HPE VME] VirtIO driver injection - version used

Post by cody.ault »

Thanks, I'll check with the team. Was there a support case for this issue by chance?
JaySt
Service Provider
Posts: 549
Liked: 126 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: [HPE VME] VirtIO driver injection - version used

Post by JaySt »

No support case yet. We were notified of the possible issue and fix(downgrade) and just tried that first (successfully it seems)
Veeam Certified Engineer
cody.ault
Veeam Software
Posts: 179
Liked: 89 times
Joined: Nov 04, 2010 2:53 pm
Full Name: Cody Ault
Contact:

Re: [HPE VME] VirtIO driver injection - version used

Post by cody.ault »

Ok. Thank you for the info. I've let our QA team know and they are looking into it.
marco.mosca
Service Provider
Posts: 5
Liked: 2 times
Joined: Feb 06, 2023 1:18 pm
Full Name: Marco Mosca
Contact:

Re: [HPE VME] VirtIO driver injection - version used

Post by marco.mosca »

Hi Cody,

We can confirm we are looking at the same area from a different angle. We opened Veeam Support Case #08193094 because on restore to VME, VBR reports "VirtIO driver injection required" and runs virt-v2v-in-place even when the guest OS already has the VirtIO drivers installed — so the injection is not conditional on the drivers being present or absent.

The summary of what support confirmed to us:
- The exact criteria used to display the "VirtIO driver injection required" message are not documented, and current observations indicate that pre-existing VirtIO drivers do not prevent the injection workflow from running.
- There is no documented or supported way to skip or disable driver injection for KVM restores.
- There is no documentation identifying the bundled virtio-win version, no supported way to pin a specific version, and no logging that records the installed driver version.
- The supported approach is to run the restore through Veeam and let the full virt-v2v-in-place workflow, driver injection included, complete as designed.

Combined with what @JayST is reporting, this is exactly the risk scenario: if the bundled virtio-win build carries a known storage I/O regression (kvm-guest-drivers-windows issue #1453), every restore to VME silently ships that driver into the guest, there is no log entry telling us which version was injected, and the only remediation is post-restore manual rollback inside the guest — on a multi-TB SQL VM, after the corruption has already happened.

From an SP perspective this makes restore results non-deterministic across VBR builds, which is hard to reconcile with a tested/validated DR procedure.

What we would like to see:
- The injected virtio-win version written to the restore session log (and ideally shown in the session UI).
- Documentation of which virtio-win build is bundled with each VBR build/patch.
- A supported way to supply a custom virtio-win source (e.g. a specific ISO/driver path), or at least to pin the version.
- A supported option to skip injection when the guest already has working VirtIO drivers — which would also considerably shorten restores of large VMs.

@JaySt: could you confirm which VBR build you restored with, and whether you identified the injected version only from the driver properties inside the guest after the restore? We would like to correlate it with what we see on our side.

Thanks
Marco
JaySt
Service Provider
Posts: 549
Liked: 126 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: [HPE VME] VirtIO driver injection - version used

Post by JaySt »

we used v13.1, the first version with the injection capabilities. 13.1.0.411
Yes, we confirmed the version by checking the driver properties within the guest after the restore. 100.102.104.29000 in our case.

+1 for the ability to disable injection as a simple quick enhancement. However, i wonder if injection currently queries the existence for virtio drivers during restore preventing recplacement with what Veeam ships ?
Veeam Certified Engineer
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest