-
JaySt
- Service Provider
- Posts: 549
- Liked: 126 times
- Joined: Jun 09, 2015 7:08 pm
- Full Name: JaySt
- Contact:
[HPE VME] VirtIO driver injection - version used
we got some feedback after trying the restore of a multi TB SQL database VM to HPE VME, now that we have the VirtIO driver injection with v13.1
injection seems to use storage driver 100.102.104.29000
Injected during restore of a Windows Server 2019 server.
However, we immediately saw database corruptions, storage io issues.
Seems that there is a known storage IO issue with the latest VirtIO drivers : https://github.com/virtio-win/kvm-guest ... ssues/1453
Reverting the driver to what's included in VirtIO 0.1.271 (the version just before the latest 0.1.285) seems to have fixed the storage IO error.
the issues seems to be fixed on github, but not yet merged into the latest stable. I think Veeam uses a version still affected by the issue.
any thoughts?
injection seems to use storage driver 100.102.104.29000
Injected during restore of a Windows Server 2019 server.
However, we immediately saw database corruptions, storage io issues.
Seems that there is a known storage IO issue with the latest VirtIO drivers : https://github.com/virtio-win/kvm-guest ... ssues/1453
Reverting the driver to what's included in VirtIO 0.1.271 (the version just before the latest 0.1.285) seems to have fixed the storage IO error.
the issues seems to be fixed on github, but not yet merged into the latest stable. I think Veeam uses a version still affected by the issue.
any thoughts?
Veeam Certified Engineer
-
cody.ault
- Veeam Software
- Posts: 179
- Liked: 89 times
- Joined: Nov 04, 2010 2:53 pm
- Full Name: Cody Ault
- Contact:
Re: [HPE VME] VirtIO driver injection - version used
Thanks, I'll check with the team. Was there a support case for this issue by chance?
-
JaySt
- Service Provider
- Posts: 549
- Liked: 126 times
- Joined: Jun 09, 2015 7:08 pm
- Full Name: JaySt
- Contact:
Re: [HPE VME] VirtIO driver injection - version used
No support case yet. We were notified of the possible issue and fix(downgrade) and just tried that first (successfully it seems)
Veeam Certified Engineer
-
cody.ault
- Veeam Software
- Posts: 179
- Liked: 89 times
- Joined: Nov 04, 2010 2:53 pm
- Full Name: Cody Ault
- Contact:
Re: [HPE VME] VirtIO driver injection - version used
Ok. Thank you for the info. I've let our QA team know and they are looking into it.
-
marco.mosca
- Service Provider
- Posts: 5
- Liked: 2 times
- Joined: Feb 06, 2023 1:18 pm
- Full Name: Marco Mosca
- Contact:
Re: [HPE VME] VirtIO driver injection - version used
Hi Cody,
We can confirm we are looking at the same area from a different angle. We opened Veeam Support Case #08193094 because on restore to VME, VBR reports "VirtIO driver injection required" and runs virt-v2v-in-place even when the guest OS already has the VirtIO drivers installed — so the injection is not conditional on the drivers being present or absent.
The summary of what support confirmed to us:
- The exact criteria used to display the "VirtIO driver injection required" message are not documented, and current observations indicate that pre-existing VirtIO drivers do not prevent the injection workflow from running.
- There is no documented or supported way to skip or disable driver injection for KVM restores.
- There is no documentation identifying the bundled virtio-win version, no supported way to pin a specific version, and no logging that records the installed driver version.
- The supported approach is to run the restore through Veeam and let the full virt-v2v-in-place workflow, driver injection included, complete as designed.
Combined with what @JayST is reporting, this is exactly the risk scenario: if the bundled virtio-win build carries a known storage I/O regression (kvm-guest-drivers-windows issue #1453), every restore to VME silently ships that driver into the guest, there is no log entry telling us which version was injected, and the only remediation is post-restore manual rollback inside the guest — on a multi-TB SQL VM, after the corruption has already happened.
From an SP perspective this makes restore results non-deterministic across VBR builds, which is hard to reconcile with a tested/validated DR procedure.
What we would like to see:
- The injected virtio-win version written to the restore session log (and ideally shown in the session UI).
- Documentation of which virtio-win build is bundled with each VBR build/patch.
- A supported way to supply a custom virtio-win source (e.g. a specific ISO/driver path), or at least to pin the version.
- A supported option to skip injection when the guest already has working VirtIO drivers — which would also considerably shorten restores of large VMs.
@JaySt: could you confirm which VBR build you restored with, and whether you identified the injected version only from the driver properties inside the guest after the restore? We would like to correlate it with what we see on our side.
Thanks
Marco
We can confirm we are looking at the same area from a different angle. We opened Veeam Support Case #08193094 because on restore to VME, VBR reports "VirtIO driver injection required" and runs virt-v2v-in-place even when the guest OS already has the VirtIO drivers installed — so the injection is not conditional on the drivers being present or absent.
The summary of what support confirmed to us:
- The exact criteria used to display the "VirtIO driver injection required" message are not documented, and current observations indicate that pre-existing VirtIO drivers do not prevent the injection workflow from running.
- There is no documented or supported way to skip or disable driver injection for KVM restores.
- There is no documentation identifying the bundled virtio-win version, no supported way to pin a specific version, and no logging that records the installed driver version.
- The supported approach is to run the restore through Veeam and let the full virt-v2v-in-place workflow, driver injection included, complete as designed.
Combined with what @JayST is reporting, this is exactly the risk scenario: if the bundled virtio-win build carries a known storage I/O regression (kvm-guest-drivers-windows issue #1453), every restore to VME silently ships that driver into the guest, there is no log entry telling us which version was injected, and the only remediation is post-restore manual rollback inside the guest — on a multi-TB SQL VM, after the corruption has already happened.
From an SP perspective this makes restore results non-deterministic across VBR builds, which is hard to reconcile with a tested/validated DR procedure.
What we would like to see:
- The injected virtio-win version written to the restore session log (and ideally shown in the session UI).
- Documentation of which virtio-win build is bundled with each VBR build/patch.
- A supported way to supply a custom virtio-win source (e.g. a specific ISO/driver path), or at least to pin the version.
- A supported option to skip injection when the guest already has working VirtIO drivers — which would also considerably shorten restores of large VMs.
@JaySt: could you confirm which VBR build you restored with, and whether you identified the injected version only from the driver properties inside the guest after the restore? We would like to correlate it with what we see on our side.
Thanks
Marco
-
JaySt
- Service Provider
- Posts: 549
- Liked: 126 times
- Joined: Jun 09, 2015 7:08 pm
- Full Name: JaySt
- Contact:
Re: [HPE VME] VirtIO driver injection - version used
we used v13.1, the first version with the injection capabilities. 13.1.0.411
Yes, we confirmed the version by checking the driver properties within the guest after the restore. 100.102.104.29000 in our case.
+1 for the ability to disable injection as a simple quick enhancement. However, i wonder if injection currently queries the existence for virtio drivers during restore preventing recplacement with what Veeam ships ?
Yes, we confirmed the version by checking the driver properties within the guest after the restore. 100.102.104.29000 in our case.
+1 for the ability to disable injection as a simple quick enhancement. However, i wonder if injection currently queries the existence for virtio drivers during restore preventing recplacement with what Veeam ships ?
Veeam Certified Engineer
Who is online
Users browsing this forum: No registered users and 1 guest