Direct SAN Access Performance - EQL PS4100

[leebtish]

Hi all,

We've been long time users of Veeam and love the product. We have recently moved our Veeam backup server (9.5 U3) to a DELL R710 with local storage (H700 Controller, RAID 10 stripe size 64k). Server has 2 dedicated Broadcom NICs (5709) connected to 1GB SAN Fabric using DELL Powerconnect Switches with PS4100 arrays. Local storage is formatted with REFS at 64k and all fine. Firmware on server is up to date and OS is Windows Server 2016 Standard.

Ran the first job last night and we're seeing speeds averaging 83Mbps in the job which is using SAN as the backup mode but I think we should be seeing better performance. Now this is the first run of the job so it's creating new backup files which could be the issue but I ran a test job twice last night and no difference. I've read the sticky's on Direct SAN Access and many of the threads on these forums and some mention not using MPIO.

I have installed the DELL MPIO plugin which looks to be functioning correctly and is using least queue depth. Before disabling MPIO I was wondering if anyone had any advice or perhaps the performance we are seeing is acceptable for 1GB SAN ? Job stats show - Source 99% > Proxy 6% > Network 5% > Target 0%

I didn't want to change autotuning for the nic's as recommended by some before getting an opinion if possible on the forums.

Thanks in advance.

Lee

[s_t]

So the PS4100 is your production storage acting as source for the backup, not as target?

iSCSI Storage and especially DELL EqualLogic is really latency sensitive for single-thread file access

Without tuning your network adapter, you could optimize it a little bit by increasing the number of parallel tasks, try to increase maximum allowed parallel tasks for proxy as well as repository role. First make sure that cpu usage is currently not high during Backup, as high cpu load could also be a reason for your “proxy 99%” beside I/o wait

I would recommend to also tune your network interfaces by disabling delayed-ack/nagles algorithm. You should find this in the EqualLogic Manual and in the best practices

“To disable delayed ACK and Nagle’s algorithm, create the following entries for each SAN interface subkey in the Windows Server 2012 registry:

Subkey location
HKEY_LOCAL_MACHINE \ SYSTEM \ CurrentControlSet \ Services \ Tcpip \ Parameters \ Interfaces \ <SAN interface GUID>

Entries
TcpAckFrequency
TCPNoDelay

Value type
REG_DWORD”

Set both to “1”

(Registry keys are the same for Server 2008/2008r2/2012/2016)

For typical Server workload with massive parallel i/o this should have no big effect, but for single-thread sequential read we had massive speed gains for some customers.

The names of the registry keys are case-sensitive (and wrong in some manuals)


You may also have a look at the other best-practices for you Storage at http://en.community.dell.com/techcenter ... -documents

[leebtish]

Hi Sebastian,

Thank for the quick response. Correct, the PS4100 is the source. Our target has internal local storage. The CPU usage on the server is low and the proxy rarely goes above 6%. The source is always 99%.

The PS4100 was configured as per DELL best practices and our powerconnect switches were configured by DELL support. I will however check the documentation again regarding the delayed ack setting. I will experiment with enabling parallel processing and also the registry settings as suggested.

The last job was incremental and ran at 65MB/s so it's degraded further which doesn't make sense given the first backup was faster and it was a full.

Thanks

Lee