Discussions specific to the VMware vSphere hypervisor
Post Reply
DJSnels
Lurker
Posts: 2
Liked: never
Joined: Mar 06, 2014 7:56 am
Contact:

Backup VMs with Passthrough PCIe devices (GPU)

Post by DJSnels »

I was wondering if there is a method to backup VMWare guests that have a PCIe device passthrough (such as in our case a GPU).
Backing up such VM failes because VMs with passthrough devices cannot be snapshotted. Is there any way to skip the snapshot and backup the VM using VSS only?
I prefere NOT to shut down the guest.

ESX: 5.5
Veeam: 7
Guests: Server 2012R2

veremin
Product Manager
Posts: 17777
Liked: 1625 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by veremin »

Passthrough disks are automatically skipped from processing, since they can't be snapshotted. Do you mean you want to backup VMs on a file level? Thanks.

dellock6
Veeam Software
Posts: 5918
Liked: 1739 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by dellock6 »

To clarify Vladimir, a passthrough device is different from a passthrough disk, VMware calls this DirectPath I/O. In this case it's the graphic processor passed to the VM directly.
Since this process completely by-passed th VMM (virtual machine manager) and let's the guest os use directly the hardware device, I suspect it's a limit that cannot be avoided when calling VADP libraries. Probably some lab tests are needed, but my feeling the problem cannot be avoided (just like for example vmotion is not supported for obvious reasons)

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

dellock6
Veeam Software
Posts: 5918
Liked: 1739 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by dellock6 »

I finally found the answer, sadly for the opener, a VM with DirectPath I/O enabled cannot be saved with VADP libraries:

http://pubs.vmware.com/vsphere-50/index ... 88613.html

Look at the 3rd bullet point. I'm not even sure it's enough to power off the VM.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

lp@albersdruck.de
Enthusiast
Posts: 82
Liked: 33 times
Joined: Mar 25, 2013 7:37 pm
Full Name: Lars Pisanec
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by lp@albersdruck.de »

dellock6 wrote:I finally found the answer, sadly for the opener, a VM with DirectPath I/O enabled cannot be saved with VADP libraries:

http://pubs.vmware.com/vsphere-50/index ... 88613.html

Look at the 3rd bullet point. I'm not even sure it's enough to power off the VM.

Luca.
I have a VM with a DirectPath I/O PCIe passthrough (LSI SAS Controller). I can backup the VM without problems as long as it is powered off. Even CBT works.

dellock6
Veeam Software
Posts: 5918
Liked: 1739 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by dellock6 »

Hi Lars, thanks for the additional information, really appreciated.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

DJSnels
Lurker
Posts: 2
Liked: never
Joined: Mar 06, 2014 7:56 am
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by DJSnels »

Thanks for the replies.
Is there any way to create Veeam backups without the use of snapshots? e.g. Microsoft VSS only?
I don't like using any other software (such as Windows Backup) to backup VM's with passthrough PCI devices.

dellock6
Veeam Software
Posts: 5918
Liked: 1739 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by dellock6 »

Veeam does not supports backups at the guest level, they are always done at the hypervisor level using VADP libraries, and thus VMware snapshots.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2020
Veeam VMCE #1

Vitaliy S.
Product Manager
Posts: 24075
Liked: 1807 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by Vitaliy S. »

DJSnels wrote:Is there any way to create Veeam backups without the use of snapshots? e.g. Microsoft VSS only?
No, VM snapshot is still required.

leedudmish
Influencer
Posts: 11
Liked: 2 times
Joined: May 09, 2016 6:13 pm
Full Name: Lee
Contact:

[MERGED] Veeam Backup VM with Passthrough Device

Post by leedudmish »

Hi All,

We are looking at gpu passthrough for VM's for high end users and have the lab working ok, during setup it did state snapshots are not supported with machines that have a passthrough device.

Can this be confirmed or is this something which veeam can overcome, is there any pre/post scripts whcih could be run to alleviate the issue at all such as shut down the vm, take snapshot, then switch vm back on again after the backup.

The vm's will be running windows 10 and are end user based so we can switch off if necessary?

thanks

Lee

veremin
Product Manager
Posts: 17777
Liked: 1625 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by veremin »

Your post has been merged into existing discussion. Kindly, check the answers provided above.

Additionally, you can try to backup such VMs from the inside, using Veeam Endpoint Backup.

Thanks.

umenz
Enthusiast
Posts: 52
Liked: never
Joined: Jun 15, 2011 9:24 am
Full Name: Ulrich Menz
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by umenz »

Hi All,

Long time passed, any news about this Topic with VMware 6.5 or 6.7 and Passthrough GPU ?

thanks

Ulrich

danswartz
Expert
Posts: 215
Liked: 28 times
Joined: Apr 26, 2013 4:53 pm
Full Name: Dan Swartzendruber
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by danswartz »

I would dearly love to be able to backup VMs with VMDirectPath devices, but apparently that is a no go. I'd like to pass through 40gbe enet adapter to my 2 win 2012r2 veeam proxies to improve performance, but no luck. I suppose I could install the veeam backup for windows service on those guests?

foggy
Veeam Software
Posts: 19285
Liked: 1741 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by foggy »

Yes, you can use Veeam Agent for Windows.

danswartz
Expert
Posts: 215
Liked: 28 times
Joined: Apr 26, 2013 4:53 pm
Full Name: Dan Swartzendruber
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by danswartz »

Thanks. Waiting for new mellanox cards to arrive to try this :)

MSc
Influencer
Posts: 14
Liked: 2 times
Joined: Jan 19, 2012 2:46 pm
Full Name: Martin Schenker
Location: Germany
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by MSc »

Any news on this?

We're trying to add a NVIDIA GPU card (Quadro P2000) via passthrough to a Win2019 server. ESXi is 6.7 U1... still only "Veeam Agent for Windows" as the only backup solution here?

Cheers, Martin

Gostev
SVP, Product Management
Posts: 26460
Liked: 4143 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Backup VMs with Passthrough PCIe devices (GPU)

Post by Gostev »

Yes. Also, the news would come from VMware, not from Veeam - as the limitation is on vSphere side.

Post Reply

Who is online

Users browsing this forum: Google [Bot], johnsonew, Vitaliy S. and 31 guests