NVME-oF support for vSphere 7 with Netapp storage

[alexandre.rogoski]

Can you allow me to access that page?

[Mildur]

Hi Alexandre

This page is an internal one and can only be accessed by veeam accounts.

Thanks
Fabian

[Butha]

Morning all - as many times before - it's a frustrating place so sit as an end user where between two large vendors, one the storage (Netapp) and one the company you have used and trusted for 10 years with all your data - cannot see eye to eye or provide clear answers to questions asked years before.

We are about to refresh our complete storage estate with new NVME arrays (Netapp ASA) - and as a last thought I checked support for ASA. The Veeam documentation doesn't list it. It's been a backwards and forwards thing between Netapp and Veeam (Support calls logged boths sides) _ with a lot of "Should, perhaps.. the ASA is just a subset of AFF etc etc" but nobody can agree. Second issue is the support for iSER (iscsi extensions over RDMA) - it's been around on Vmware since 6.7 - and anything from a cheap NAS to high end SAN's support it. Once again - Veeam cannot answer if it will work, if the storage API's etc are the same (it's the same Ontap) - so again an assumption that it " Might/should" work. More bigger issue - Netapp doesn't support iSER on any high end models running Ontap - but the DO support NVME/TCP (Which has been supported on Vmware for a while now). I see NVME/TCP support was requested in 2020 already here, but just last week the comments around it is still " not yet/cannot disclose".

How do we get an honest open answer to the question? Has it been tested? Is it on the roadmap? What is the ETA more or less on it? Will V12 for example fully support it? etc.

An answer on iSER and NVME/tcp support from Veeam is required asap please - and if not, either a change to new backup vendor, or different storage vendor or combination of is the only option it seems.

@mildur - please enlighten us to what that internal document says if you can.

[rennerstefan]

Hi everyone on this thread,

on the last post it was asked to give some light on this topic.
Today, as already mentioned, NVMe-oF is not supported with Veeam's Storage Integration but it is something we look into closely.

Most storage vendors started to implement NVMe-oF for FC a while ago but TCP support only came for some of them within the last 12 month or for most only the last 6 month.

To better understand the different scenarios from you, I have some questions to everyone here and the ones adding feedback going forward:
- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe?
- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? (even if this thread is called "with NetApp" I expect others to read as well)
- Is NVMe-oF in your case used for regular VMware data stores?
- Is there anything outside VMware you use NVMe-oF with?
- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? (https://core.vmware.com/resource/whats- ... 21636-sub1)

I can't comment on the timeline or give a ETA at this point but as mentioned above it for sure is something we look into.

Any feedback is welcome.

Thanks

[segfault]

Here goes:

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
FC (w/ NetApp)


- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe?
Yes. 2-3 years (based on our existing lifecycle replacement cycle)

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with?
NetApp (today). May add Pure to the mix during the next storage refresh.


- Is NVMe-oF in your case used for regular VMware data stores?
Yes. Lower latency to the VM and better array utilization / iops from the same array hardware. Veeam is the only reason we have not flipped everything over.


- Is there anything outside VMware you use NVMe-oF with?
No.

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option?
No.

[popjls]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? - FC mostly and TCP eventually.
- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? - At some point, no rush.
- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? (even if this thread is called "with NetApp" I expect others to read as well) - Dell PowerStore T
- Is NVMe-oF in your case used for regular VMware data stores? - Yes
- Is there anything outside VMware you use NVMe-oF with? - No
- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? - At some point yes.

[sandsturm]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? - FC (with NetApp MetroCluster)
- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? - No, thats currently not planned
- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? - NetApp MetroCluster. AFF A400 and AFF A800
- Is NVMe-oF in your case used for regular VMware data stores? - Yes
- Is there anything outside VMware you use NVMe-oF with? - Currently not
- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? - No

[rennerstefan]

@sandsturm thanks for your reply. Just a small question. I recently was told that there is no support for metrocluster and NVMe-oF today which can also be found here (https://docs.netapp.com/us-en/ontap/san ... ncept.html): “MetroCluster configurations do not support front-end NVMe host connectivity.”

Do you have any information/document that it changed?

[sandsturm]

I know it's currently not supported on MetroCluster, hope this will change soon. But I can't tell you, if and when this will change. We are using MetroCluster and non-MetroCluster systems in our company and finally I want to use NVMe-oF on all our systems (if possible and supported).

thx,
sandsturm

[rennerstefan]

Ok thanks for clarifying. Now we are on the same page.

[ToddMenefee]

Our company has begun to implement NVMe but we have reached a roadblock due to Veeam lagging behind on current technologies. Please prioritize support for NVMe storage integration.

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? - FC


- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? -No


- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? (even if this thread is called "with NetApp" I expect others to read as well) – Pure Storage x70


- Is NVMe-oF in your case used for regular VMware data stores? - Yes


- Is there anything outside VMware you use NVMe-oF with? - No


- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? (https://core.vmware.com/resource/whats- ... 21636-sub1) - Possibly

[Butha]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? TCP&RDMA

- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? TCP Already

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? (even if this thread is called "with NetApp" I expect others to read as well) Netapp

- Is NVMe-oF in your case used for regular VMware data stores? - Not yet as support from Veeam is required.

- Is there anything outside VMware you use NVMe-oF with? No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? No - main use case is leveraging RDMA which has been present in the HBA's (Mellanox) and switches (Nexus) for a while but unused - to improve throughput, reduce latency etc.

[Eric from Paris]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? FC

- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? No

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? Netapp

- Is NVMe-oF in your case used for regular VMware data stores? - Not yet as support from Veeam is required.

- Is there anything outside VMware you use NVMe-oF with? No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? No

[ASG]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
FC

- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe?
Currently not planned

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? (even if this thread is called "with NetApp" I expect others to read as well)
IBM HyperSwap Cluster

- Is NVMe-oF in your case used for regular VMware data stores?
Yes

- Is there anything outside VMware you use NVMe-oF with?
No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? (https://core.vmware.com/resource/whats- ... 21636-sub1)
No

[nlsrkr]

@sandsturm thanks for your reply. Just a small question. I recently was told that there is no support for metrocluster and NVMe-oF today which can also be found here (https://docs.netapp.com/us-en/ontap/san ... ncept.html): “MetroCluster configurations do not support front-end NVMe host connectivity.”

Do you have any information/document that it changed?

The above is addressed in ONTAP 9.12.1 for MetroCluster IP configurations as can be seen in the release notes:
https://library.netapp.com/ecm/ecm_down ... MLP2492508
page 52.

The ONTAP documentation is being updated soon.

Cheers

[sandsturm]

The above is addressed in ONTAP 9.12.1 for MetroCluster IP configurations as can be seen in the release notes:
https://library.netapp.com/ecm/ecm_down ... MLP2492508
page 52.

That sounds fantastic, I'm sure you will support ONTAP 9.12.x including NVMe-oF with Veeam V12 then?

thx,
sandsturm

[sandsturm]

Just to bring this topic up again: Does anyone know, if NVMe-oF is supported now with VBR v12? I couldn't find something in "Whats new" document?

[rennerstefan]

Hi,
no NVMe-oF support is not part if v12.
It is still something we are considering for future releases so feedback is still welcome.
Thanks

[sandsturm]

Hi Stefan

thanks for the answer. I understand that something like "Direct-NVMe-oF" (comparable to DirectSAN) as veeam transport mechanism is not an option in this case. But is it unsupported if I configure my ESXi hosts and my storage system for NVMe-oF and just use another Veeam-transport mechanism (not a Direct-anything transport mechano) to create backups of VMware vitual machines running on such NVMe-oF datastores?

thx,
sandsturm

[rennerstefan]

Hi again,

No that is not unsupported.
Any other mode than direct san or BfSS just uses the VMware api to retrieve data.
That said hot-add and nbd will be fine.
We don’t claim dedicated support for that as it is part of VMware api stack we use.

This forum thread is to understand the demand mainly for our storage integration.

Hope that helps.

[blackstone]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
RMDA

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with?
PureStorage

- Is NVMe-oF in your case used for regular VMware data stores?
Yes

- Is there anything outside VMware you use NVMe-oF with?
No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option?
No

[SkyDiver79]

I have any Customers with DELL Powerstore and NVMe over TCP.
Now we use NBD in most cases, but a Linux Proxy with direct SAN would be speed up the Backups.

[MatzeB]

+1 for this Request from my side.
At the moment, our customers running almost >90% NFS for VMware in their Netapp environments. Due to the new hardware (End-to-End NVME) and the performance advantages of NVME-oF I could imagine that this will be our "new NFS", so our new default. Since most customers use NFS today, I think the focus will be on NVME over TCP/IP.

As a simple workaround we would also use HotAdd/NBD - but a real storage integration would be much better.


What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
FC or TCP

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with?
Netapp Ontap

- Is NVMe-oF in your case used for regular VMware data stores?
Yes we plan to

- Is there anything outside VMware you use NVMe-oF with?
No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option?
No



Regards
Matze

[p.lan]

We have deployed several NVMe-oF arrays with NVMe/TCP configured for our customers and we see it trending this direction quite a bit. At this rate, we will probably be deploying more NVMe/TCP than iSCSI with hosts/storage as of end of 2023 and would really love for Veeam to support the storage integration for NVMe. At the very least: support integration with the storage and vmware hosts to mount/dismount storage snapshots for instant recovery from snapshots on NVMe volumes.

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? TCP

- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? Already on TCP

- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? Dell Powerstore, Netapp,

- Is NVMe-oF in your case used for regular VMware data stores? - Yes

- Is there anything outside VMware you use NVMe-oF with? No

- With the recent VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? No

[runste]

+1 on this request.

Any news on when Veeam will support NVME-oF?
We need it for IBM Flashsystem.

[rennerstefan]

Thanks for your feedback.
This is something we are actively researching for after 12a.

Thanks

[segfault]

This is something we are actively researching for after 12a

As a customer who has been waiting a very long time for this, responses like this are getting a bit frustrating at this point.

This thread is over three years old now. To put that in context, v10 was out when the thread was created. v10a then came and went. Then v11. Then v11a. Now v12.

Now it sounds like it will not be any sooner than v13 as you just killed any hope of it making the v12a cut. So maybe late 2024 at best case now?

We really like the direct SAN access route since it is so fast compared to just about every other option we have, plus it keeps the time that the VM has an active snapshot applied down to an absolute minimum. We don't want to give that up but our database and app teams teams are really eyeballing up the latency reduction that we get by bumping to a full end to end NVMe system. This has been on the market since ONTAP 9.4 which was released back in 2018 and fully supported by every other component in our virtualization stack for the past 4+ years except for Veeam.

[Brian_Larsen]

+ 1 for NVME over TCP support.

[dbewernick]

Hey @Brian_Larsen,
would you mind providing some additional information?
- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA?
- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe?
- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with?
- Is NVMe-oF in your case used for regular VMware data stores?
- Is there anything outside VMware you use NVMe-oF with?
- With the VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option?

[Brian_Larsen]

- What is the NVMe-oF Fabric used in your implementation(s) - FC, TCP, RDMA? NVMe over TCP
- If you use NVMe-oF FC today, do you expect to switch to TCP and if yes what is the timeframe? Don't use NVME-oF today.
- What is the Storage Vendor and Array type you use or plan to use NVMe-oF with? Dell PowerStore 500 T.
- Is NVMe-oF in your case used for regular VMware data stores? Yes.
- Is there anything outside VMware you use NVMe-oF with? Not so far.
- With the VMware announcement to support vVols with NVMe-oF, do you expect to leverage this new option? No.