SQL Box Replication and Network Drop Outs

[adverse]

Hello,

We are using Veeam to replicate an MSSQL VM but we're getting very slow processing rates. We also see network drop outs when the snapshots are removed at the end of the process.

I appreciate this question has been asked countless times, and I have spent a good few hours trawling through the forum responses, but there appears to be so many variables I'm not sure what actually applies in my specific situation. I'm new to both VMWare and Veeam.

Details:

VM I'm replicating:
Windows 2008 R2
MSSQL 2005
410Gb total size - 25Gb OS Drive, 180Gb SQL data and logs drive, 115Gb SQL backups drive (both db and tran log backups), 90Gb demo environment
As well as the database changes themselves, there are approx 100Gb changes between replications because of the large SQL backup files on the 115Gb disk. This is the disk that the replication seems to hang on for the majority of the job.

VMWare Environment
VMWare ESXi 4
We have two clusters which exist in the same VMWare Datacentre, both with their own SAN. These are in two different buildings on the same site. The buildings are connected by a 1Gb fibre link, which is around 200 meters in length.

Backup Server (this is a VM on the source host)
Veeam 4.1.1
Windows 2008 R2
SQL 2005 Express
4 x vCPU
4Gb RAM
Veeam 4.1.1
Veeam replication Mode is VMWare vStorage API - SAN with failover
CPU and RAM utilisation is low when replication job is running


What's happening:
I did an initial replication over the weekend, when there was no processing on the box, which completed successfully in around 50hrs. The average processing rate was 2Mb/s. We have not replicated anything approaching this size before, but have noted a server we replicated that was 140Gb also saw this processing rate when it initially replicated, but now on average does around 40MB/s. Smaller boxes appear to be around 50-80MB/s, and we have one box doing 300+MB/s (a filestore).

Overnight (23:00) the SQL backups run, which generate approx 100Gb of new backup files.

The replication of the box is set to run every hour between midnight and 22:30. The replication that starts at midnight is still running at 10am, at 2MB/s, with reported completion time of 2hrs 30mins. I've so far had to kill every one, as it massively effects performance of the server. After killing the job, the snapshot takes around 20mins to delete and causes the two applications that rely on this SQL server to fall over due to what seems to be some form of network issue.

Things I've tried:
Removed the 90Gb demo environment drive, taking total size to 320Gb. Same issue.
Removed the 180Gb SQL Data and Logs drive (so only replicating OS and backups), taking size to 140Gb. Same issue.
Transferred a 6.5Gb file using explorer from the SQL VM to the Veeam server - 30Mb/s. These are on the same host in the same cluster, using the same SAN.
Transferred the same file from Veeam server to destination cluster, using vSphere client pointed at remote host's IP Address - 1.86MB/s
Transferred same file using explorer from Veeam Server to machine on the network at the destination cluster's site - 9MB/s.

I would like to try other replication modes to see if these help, but I'm not sure what to go for (Virtual Appliance Mode perhaps?). The problem is that I can't run a test and stop it during business hours (06:00-22:30) because of the network drop out effect stopping the job has. Therefore I'm limited to one scheduled test per night.

Any comment or advice would be greatly appreciated.

Thanks,

Adam.

[foggy]

First, I would strongly recommend upgrading to our last Veeam B&R 5.0.2 version.

Secondly, performance bottlenecks can be caused by three main reasons described in this thread. Make sure you have your backup server configured properly to run in Direct SAN mode so it does not fail over to a Network mode. You can find some configuration guidelines here (remember that FC SAN does require physical Veeam server for direct SAN access).

[adverse]

Thanks very much for your reply, means I have something to aim now which is great. I'll give these a go and see how I get on.