Veeam 9.5 - "Direct SAN" mode limited at 8gbps

[JuA]

Hi,

Environment:

• * Veeam 6.5
  * VMware 6.5
  * Netapp AFF300 via Fibre Channel
  * Physical Proxy vith Fibre Channel HBA
  * All Fibre Channel gear @16gbps.
  * No Backup I/O Control.

I am deploying a physical proxy , with internal disks as primary backup storage. Everything is connected fine to perform backups from SAN storage snapshots. Everything in the SAN is at 16gbps.

I see that my job is performing (reading) at ~950MBps = 76800mbps ~= 8gpbs.
In the same time, the proxy is writing on the internal disks at ~370MBps. I don't see contention on the local disks.

Veeam says that the primary bottleneck is the source (Source 99% > Proxy 21% > Network 52% > Target 3%).

I can't find the origin of the limit.

I tried to put a very high bandwidth limit at the Proxy level, but the GUI accepts a maximum of 1024 MBps...

Does the proxy limit the trafic speed at 1024 MBps ? if yes, is it possible to change this limit ?

Any idea is welcome.

[ctg49]

What's your disk type? If it's not flash, you're probably limited by the SAN disk speed. If it is flash, i'd be a little more curious as you should be saturating a 16gbps connection unless there's something wonky going on internally (controller flooded? SAN CPU capped out?)

[JuA]

All the Netapp array from the "AFF series" are full flash.

[Andreas Neufert]

It is hard to say, but maybe you can create a seperate LUN and mount it to the Veeam Server and use a disk performance tool (1MB block sizes random read) to check if you get faster backups.
So that you have a reference point to see if it is Veeam or the actual configuration.

Maybe a more simple task would be to copy files arround and look at the speed there (multiple in parallel).

[Andreas Neufert]

And please check if you use Veeam 9.5 Update 3a (6.5 is very old).

[JuA]

It's 9.5 last update, of course!

I wrote the title of my message from a copy/past when I searched the forum before asking... and I I didn't change the version.

Can you change the title and my original post ? If no, I will close the post and open a new one.

[Andreas Neufert]

Done, as well please tell us how many disks you backup in parallel. The maximum speed can be reached only with multiple (arround 20 disks) parallel backups.
As well please check with Active Full processing speed first.

[JuA]

For this fist test, we have only 1 VM in the job, and nothing else is running.

It has 4 disks, but 3 of them are quite small, so after some seconds, there is only one disk processed.

[JuA]

New test: I put a second VM in my job, so now I have 2 disks being read in parallel. 1,5GBps.

[nitramd]

Have you considered adding vCenter to your test job? It usually has 10-12 disks.

[skrause]

Are you sure all of your FC SFPs are actually 16gb and not 8gb?

[Andreas Neufert]

A single VM disk read stream is usually limited by the storages and they perform much better with multiple disk backups in parallel.
As well if you have only one disk that is just some 100GB, you will not reach full speed as it is over before it has started to transport data.

My recommendation is to backup at least 20 VM disks in parallel to reach full speed of the backup and storage solutions.
Btw. same apply if you work with a single VM vs. 20 VMs in parallel on primary side and transport data there.

[JuA]

While backuping 4 VM with a total of 6 disks in parallel, I was able to see the Proxy read at 2GBps = 16gbps, the nominal speed of my FC links.

So... Everything is ok.

[Andreas Neufert]

Thanks for following up and sharing this.