Improve performance?

[MS84]

Hi,
I have the following setup:

ESXI 6.7 host:
Lenovo SR650, Intel Gold 6144, 192GB RAM
SAN: Lenovo DE4000F SSD only
Veeam 10 installed on a virtual machine with Windows Server 2012R2, 4cpu, 24gb RAM

NAS:
Synology RS812+ with RX415 expansion
8x6TB WD Red
NFS share

The ESXI and NAS is located on 2 different sites with a 500mbit MPLS connection. My performance results from an active full backup is:

Processing rate: 33 MB/sec
Bottleneck: Target
Load:
Source: 2%
Proxy: 19%
Network: 6%
Target: 93%

Data:
Processed: 7,3TB (100%)
Read: 5,3TB
Transferred: 3,7TB (1,4x)

If I copy a file from my Veeam server to the NAS box I get almost 100MB/sec.



Any suggestions?

Have a nice weekend.

[soncscy]

Hey MS84,

File copies are never valid tests -- windows caching, lack of a need for proxies means that you're not really testing the actual write performance of the drive in a 1:1 way.

Couple of question:

1. Where is the proxy for the backup located, and where is the gateway for the NFS share? Sounds like you could end up with some pretty sub-optimal traffic depending on where they are. IIRC, Veeam's source agent does the deduplication, so if you have a remote proxy, then you're sending a ton of excess data over the wire, and probably then the performance starts to drop off.
2. Why NFS? It "can" be performant, but if you can do iscsi, it will always be better to provision a LUN for iscsi, and likely way more stable. Don't use the same LUN for user files you do for backups -- they're different workloads that benefit from different configurations.

iperf is a way more realistic test than a file copy, so you might try to see the sustained performance of iperf after maybe 100GB or so.

[MS84]

Hi Soncscy.

1. The proxy is the weeam server located on the same site as the ESXI host.

2. Because I read that Veeam prefere NFS over SMB and I don't have the best experience with ISCSI on a Synology NAS. It's possible for me to change to ISCSI. This NAS is ONLY used for Veeam backup repository

It's not necessary for me to know the exact speed of the link but Veeam uses 300Mbit at most and filetransfer 800MBit.

[foggy]

In addition to Harvey's considerations (which are totally right), you need a gateway server close to the storage for optimal data transfer between Veeam data movers (both during backup and restore).

[MS84]

Hi Foggy.

I don't have any hardware located on the same site as the NAS, that I can use for gateway server

Will changing from NFS til ISCSI improve my performance?

[foggy]

Likely but only the actual test will show.

[PetrM]

Hello,

I would definitely try iSCSI as Harvey and Foggy have suggested. One more possible option would be to run backup jobs locally and to transfer backups to the remote site using backup copy job. This approach will let you follow 3-2-1 rule, shorten snapshot lifetime because of faster primary backups and transfer less data to the remote site.

Thanks!

[DaveWatkins]

Running iSCSI over a 500Mb/s MPLS connection seems like asking for trouble and corruption.

[orb]

@MS84

NFS3 is latency-sensitive, you might improve may be little the situation by forcing NFS4.1

Still, I agree with the general consensus:

a proxy/gateway server on the other side should give you the best result. You can use an old workstation but you need Windows 10 for ReFS or Linux for XFS (SCSI). Here you want to leverage block-cloning technology. I don't think even with extra ram you can run VM on this model of Synology.

SCSI might do better work and allows you with ReFS/XFS and skip the full of the wire still don't expect blazing performance. We have a customer with the constraint of running iSCSI over 1Gbit/s MPLS and write operations oscillate between 30-50MB/s (ReFS)

Oli