Ultra slow performance replicating vm

[isaako]

Hi,

I'm running VeemBackup in a physical server 4GB RAM, Dual Xeon WIndows 2008 R2 64bits.
I'm running iSCSI with multipath and Equallogic PS6000XV.
I've two ESXi and the veembackup server attached to the SAN.
I can run full backups at 60MB/s, then i think that it's correctly configured.

I'm trygin this one:
ESXi-A have two LUNS LUN-1 and LUN-2. (Datastore-1 and Datatore-2)
ESXi-B have two LUNS LUN-1 and LUN-2.(Datastore-1 and Datatore-2)

I'm running a Linux VM in ESX-A in the Datastore-1. And I want to replicate this VM to ESX-B en the Datastore-2.
I'm only getting 1MB/s of proccess rate.
The connectivity between ESXi and Veeam Server is 1Gbit.
The method to do the replication is via vmWare Storage API, SAN mode only.

Can you help me?

[Vitaliy S.]

Could you please try to upload any large file from the backup server to the Datastore-2 connected to ESXi-B host using vSphere Client - Datastore Browser? What is the upload speed for this?

[isaako]

100MB in 3 minutes

[isaako]

another test directly from the veembackup server.
300Mbytes in 60seconds

[Vitaliy S.]

In this case it would be interesting to see replication job log files as well, please send those to our technical team for further troubleshooting.

[isaako]

We're evaluating the tool and i don't have any support contract. What things can i check before sending the logfiles?

[isaako]

I'm getting this in the JOBNAME_cli.log

15.09.2010 14:51:39] < 3816> alg| Data source: [BLOCKS_READER: DISK=VDDK:[testing_datastore] testingstorage/testingstorage.vmx, CTK=VSPHERE_CTK://viConn=vcenter00.es.clara.net/VM=vm-906/Snapshot=snapshot-941].
[15.09.2010 14:51:39] < 3816> alg| CPU number: [4].
[15.09.2010 14:51:39] < 3816> alg| Block digests should be calculate.
[15.09.2010 14:51:39] < 3816> alg| }
[15.09.2010 14:51:39] < 2576> dmn| Entering into cycle of asynchronous writing. Output file: [Backup of the FIB testingstorage-flat.vmdk].
[15.09.2010 14:53:26] < 3816> cli| - 0%
[15.09.2010 14:55:26] < 3816> cli| - 1%
[15.09.2010 14:57:26] < 3816> cli| - 2%
[15.09.2010 14:59:25] < 3816> cli| - 3%
[15.09.2010 15:00:01] < 4104> vim| VI SOAP connection was not used for [7] minutes and now will be cleaned up. Connection target: [XXXXXXXXXXXX]. Login: [XXXXXXX\XXXXXXXXXX].

[Vitaliy S.]

We do offer assistance during your official evaluation period, so please feel free to contact our support for help. Please forward all logs to support@veeam.com directly, we should be able to see from those what is the possibe reason for the slow speed of the replication job.

[isaako]

I've sent the log file to the support team.

Lots of thanks for your attention.

[isaako]

Hi,

The problem isn't solved.
It seems there is a 2MB limitation replicating vm. I need help.

[Gostev]

Hello, I recommend that you contact VMware and have them troubleshoot why performance of uploading files via Sphere Client Datastore Browser to ESXi-B over 1Gb LAN is so slow for you. As soon as this issue is resolved, Veeam replication speed will improve as well (as we are using the same API as the Datastore Browser). Thanks!

[isaako]

I understand that the VSphere Client it's using a File Management API, and replication must use vStorage API. Then the replication speed must be similar to backup speed.

[Gostev]

vStorage API is used to pull the source VM data for both backup, and replication. However, neither backup nor replication use vStorage API to put the data to target, instead file management APIs are used (different, depending on specific target the data is written too).

Again, as soon as you can resolve the issue causing slow upload via Datastore Browser to the same host/datastore you are replicating too, Veeam Replication speed will improve as well.

[Gostev]

Just did a quick test in my lab copying file to ESXi 4.1 (local storage) from my desktop over 1GB, and it did 12MB/s with vSphere Client on 3GB file... a few times faster than you get. Could it be the network issue (some port failed over from 1Gb to 100Mb)?

[isaako]

I've checked all the switch ports, and all are at 1000 and full duplex.
Also I've checked for errors, CRC, and port resets and all seems correctly.
This issue it's really desperate, also i've tested the iscsi access and it's fine.

Do you have more idea?

Thanks for your anwsers.