Slow replication performance ?

[JLaaij]

Hi,

Been testing replication and I hope I 've made errors setting it up.
Cause performance seem very slow to me.

Any suggestions?

Greetz Jaap

=====

Replication of NON-running VMs.

1 ESXi host v4.01u all patches

1 SAN (Starwind v5.2, W2K8 all patches)
Two Raid controllers
VMs on Raid 1: Raid0 4x WD 500Gb RE3
Replicas on Raid 2: Raid1 2x WD 500Gb RE3

Veeam v4.1.1 installed in VM with 2 vCPU with 4Gb memory

Network: 1Gb with jumbo frames and flow control.

2 VMs running: vCenter and Veeam
===

Replicating 1 VM with 2 disk total 24Gb (11,54Gb and 1,4Gb free space)

Processing rate: climbing to 12Mb
Backup mode: San with CBT

Veeam CPU: steady at 50%

Progress indication not reliable
Stayed a very long time at "Time remaining: 00:00:07"
10 minutes or so (Starwind SAN copying empty space?)

Total time for 24Gb: 45min.

Replicating 1 VM with 1 disk total 16Gb ((3,5Gb free space)

Processing rate: climbing to 12Mb
Backup mode: HOTADD with CBT

Veeam CPU: steady at 50%

Progress indication not reliable: Job summary and VM summary diffrences.
Stayed a very long time at "Time remaining: 00:00:014"
10 minutes or so (Starwind SAN copying empty space?)

Total time for 16Gb: 31min.

Replicating 1 VM with 1 disk total 16Gb (3,4Gb free space)

Processing rate: climbing to 12Mb
Backup mode: NBD with CBT

Veeam CPU: steady at 50%

Progress indication not reliable: Job summary and VM summary diffrences.
Stayed a very long time at "Time remaining: 00:00:011"
10 minutes or so (Starwind SAN copying empty space?)

Total time for 16Gb: 29min.

Network utilzation average < 5%
===

Overall spiky diskperformance

Error messages from VCenter on all jobs when starting replication:
Create virtual disk - The requested operation is not implemented by the server

[Gostev]

Hi Jaap, try running the job again and see what incremental replication speed do you get.

[JLaaij]

Hi Gostev,

Rerun the replica job on NON-running VMs. (so no changes)

Replicating 1 VM with 2 disk total 24Gb (11,54Gb and 1,4Gb free space)
-Backup mode: San with CBT
-Processing rate: 125Mb
-Total time for 24Gb: 3min15.

Also run this VM with:
-Backup mode: NBD with CBT
-Processing rate: 134Mb
-Total time for 24Gb: 3min03.

Replicating 1 VM with 1 disk total 16Gb ((3,5Gb free space)
-Backup mode: HOTADD with CBT
-Processing rate: 96Mb
-Total time for 16Gb: 2min51.

Replicating 1 VM with 1 disk total 16Gb (3,4Gb free space)
-Backup mode: NBD with CBT
-Processing rate: 122Mb
-Total time for 16Gb: 2min14.

It's seems odd that with the first VM NBD mode is faster ?

Greetz Jaap

[Gostev]

Japp, this is normal for small VMs and no changes due to some hot add mode specifics, and also because of your storage speed. If you had fibre channel SAN, and real VM workload with significant amount of changes, the results would be different

[darrellarbaugh]

I too have the issue with slow performance when using Hot Add for replication and an error in vCenter "The requested operation is not implemented by the server" when trying to create a disk when the replication starts. I see the source datastore has been attached to Veeam VM, but what is perplexing to me is the Veeam VM's nic is running at 15000 KBps... ???

I was able to P2V the server in an hour and a half, over a 1Gbps connection, surely the initial replication shouldn't take 8 hours, should it? I've tested the disks inside the VM and see that I'm able to get 150 MBps sustained throughput, when tested on the SAN.