NVME-oF support for vSphere 7 with Netapp storage

[sandsturm]

With vSphere 7 supporting NVME-oF, this becomes now more interesting to connect storagesystems to a host (vsphere hypervisor in this case), till now only some Linux derivates were supported for NVME-oF. How about the support from Veeam, if a vSphere 7 environment with Netapp storage is used and the vmware datastores are connected via NVME-oF (FC or RoCEv2) to the storage system? Does the Veeam storage integration work (and is supported) in such a scenario or not? I'm currently planning our infrastructure for next year and one of the ideas could be to go for an NVME-oF storage backend, but I can't do that if it is not supported from Veeam.

thx
sandsturm

[foggy]

Hi, nothing to share at the moment - we haven't been able to test this configuration so far. Based on what we know, NVME-oF support will require some development on our side but currently we have no particular plans regarding this feature. Thanks!

[sandsturm]

okay, thanks for your answer

[segfault]

With lower cost NVMe-oF systems coming to the market (e.g., the NetApp A250) and 32G FC being the new 16G, we are also looking to switch to NVMe-oF during our hardware replacement cycles to take advantage of the lower latency and higher overall performance offered by the newer protocol.

Is NVMe-of support on the roadmap?

[Gostev]

I don't see any reason why it would not work at least with hot add or NBD transport modes... if ESXi host is able to read data from storage, then Veeam will be able too using either of these transport modes. As for other transport modes, we can't know as we don't have access to such storage at this time.

[segfault]

I should have been more explicit: I was referring to the direct SAN transport method, similar to the current ability to mount a Fibre Channel LUN today.

I fully expect NBD modes to work, but we prefer to use the SAN route to minimize load on the hypervisor and reduce the amount of time that a snapshot is in place on the VM. We also found that using a pair of 32G FC connections to pull data off of our primary NetApp is significantly faster than trying to pull the data over a single 10G ethernet channel. (No big surprise there.)

[sandsturm]

That's exactly the thing I want too. It is clear that NBD or maybe hot add will work. But that is not a real option in a larger environment and therefore the use of a "directStorage" access possibility in case of NVMe-oF would be desirable for the future. I think, request from customers will increase in the next few months...

[Andreas Neufert]

Thanks for sharing. Indeed the NVMe-oF popps up more and more and we will look into this, but can not share timeline yet.

[sandsturm]

great to hear that, thanks!

[kaithost]

Hi, NVME-oF is "next gen SCSI" ... not isolated to Netapp, but for some other systems and more and more to come.

It would be great if this gets a seat for V12 or an update for V11.

++ requested feature ++

[EMPB]

We would definitely be very interested to have support for NVME over RDMA (as a direct SAN backup method equivalent). It is now our main connection method for VMware to PureStorage Flash X and C array.

Where is this feature on the roadmap of Veeam B&R?

[foggy]

It is on the table but without a particular ETA.

[dmitri-va]

+1 for veeam support on NVME-Of.

We also use SAN transport over iSCSI for veeam backups, and are getting a PowerStore SAN capable of NVME/ROCE and NVME/TCP.

If Veeam is going to hold back customers, who wants to continue using SAN transport, from deploying NVME-Of in vmware environments, I can see how that could become a problem. I highly advise veeam to prioritize this.

Btw, per Dell, you cannot connect to the same LUNs over different methods (NVME/TCP and iSCSI) due to different locking. But what if we connect veeam over iscsi in read-only mode.....

[foggy]

Thanks, Dmitri, the more requests the higher the priority. We're looking in this direction.

[nesj11]

+1 for veeam support on NVME-Of
thanks

[crackocain]

+1 NVMoF support for all storage vendors.

[Eric from Paris]

+1 for veeam support on NVME-Of
Thanks

[popjls]

+1 to this also.

[jjnts]

+1 for NVMEoF for NetApp

[Andreas Neufert]

Hi, I wanted to quickly follow up here. I spoke with nearly all major vendors and the feedback so far is that customers do not use NVMEoF for regular datastores but use it for specific usecases where this technology is an advantage.

Can the above commenter please check with their VMware teams if they really plan or already have integrated NVMEoF as a transport protocol between storage and the VMware hosts for regular datastores?
The vendors shared that there are many NVME storages that are connected over FC, but this is not NVMEoF.

[runste]

We use NVMW-over Fibre Channel fabric for all our VmWare ESXi dastastores.
And since Veeam 11 does not support NVME-oF (FC) we use snapshot integration for backups and present snapshots to Veeam.

This is against an IBM FlashSystem, and NVMe-oF is very fast.

[Andreas Neufert]

Hi Rune, do you mean you use Backup from Storage Snapshot or how do you "present" the snapshots to Veeam?

[runste]

Yes, "Backup from storage Snapshots".

Backup is fast and easy, restore must be done in two steps:

1. Restore to SCSI presented vmfw-datastore
2. Svmotion to original datastore afterwards.

If veeam had supported Nvme-of-presented datastore we could restore directly to original datastore

[foggy]

Hi Rune, just to confirm for our complete understanding - you're saying that backup from snapshots on volumes connected over NVME-oF works fine but for restores, you need to use SCSI volumes as an intermediate staging location?

[runste]

Yes, thats correct.
As Dimitri-va said further up,you cannot connect to the same LUNs over different methods (NVME-oF and SCSI) due to different locking.

[foggy]

Thanks for the clarification.

[segfault]

Hi, I wanted to quickly follow up here. I spoke with nearly all major vendors and the feedback so far is that customers do not use NVMEoF for regular datastores but use it for specific usecases where this technology is an advantage.

Can the above commenter please check with their VMware teams if they really plan or already have integrated NVMEoF as a transport protocol between storage and the VMware hosts for regular datastores?
The vendors shared that there are many NVME storages that are connected over FC, but this is not NVMEoF.

Are those vendors talking to their customers or just a handful of risk adverse ones that move very, very slowly?

We use NVMEoF for our regular datastores whenever we are working with an array that supports it. All of our FC switches are NVMe capable at this time (Low cost 32G FC switches were available 6+ years ago.) Same with all of the 32G HBAs we have purchased in the last 5 years. All of our newer FC arrays are also NVMe capable (NetApp A250 on the low end.) Even our A700's on the higher end support it with the newer ONTAP versions.

Compared to regular SCSI based FC, NVMe provides lower latency and less array overhead at no extra cost. What's not to like? The only drawback is that Veeam does not support it. With Veeam support, I don't see any use case where I would go with regular SCSI based vs NVMe based LUNs for our VMWare environment except in cases where there is an old array still in use.

We are keeping an eye on NVMoTCP as that will drop the cost of implementation even further. 100G network switches and NICs are cheaper than 32G or 64G FC switches unless you need the advanced FC feature set, which we do not. Ontap 9.10 supports NVMoTCP, as does VMWare 7.0U3. So we are holding off on new FC switch purchases to see how the dust settles.

[sandsturm]

Hi, I wanted to quickly follow up here. I spoke with nearly all major vendors and the feedback so far is that customers do not use NVMEoF for regular datastores but use it for specific usecases where this technology is an advantage.

Can the above commenter please check with their VMware teams if they really plan or already have integrated NVMEoF as a transport protocol between storage and the VMware hosts for regular datastores?
The vendors shared that there are many NVME storages that are connected over FC, but this is not NVMEoF.

We will definitely use NVMEoF (on a FibreChannel backend in our case) as a standard way to connect regular datastores from a Netapp system to our ESXi hosts. NVME(oF) will definitely replace SCSI as the standard protocol for the IO stack and therefore we need support from your side for this technology as soon as possible.

[ac0lada]

+1 for veeam support on NVME-Of
Not only for standalone nvme-Of access to storage, but also for cluster nvme shared access (ANA).
Thanks

One interesting behaviour: if one of the VM disks is located on traditional FC scsi datastore, all virtual machine backup will use [san], or direct storage access mode, with a physical host backup proxy.
All you need to do is to create a dummy virtual disk on a scsi datastore and you will gain access to direct storage access backup, even when all other disks and the virtual machine configuration files reside on a nvme-Of datastore. You also have to select only physical backup proxies, with access to both datastores.

[dbewernick]

+1 in relation to post461640.html#p461640