Slow Performance (again)

[antivir]

BL460c G1 (8 logical processors, 16 GB memory).
ESX 4 (update01+3 patches).
Virtual Appliance. 4 CPU for backup VM and 4 Gb memory. CBT is on.

There are slow performance (replication and backup) in my VI with any large VMs (500 GB) and any storages (EVA, MSA - 4 GB/sec, ISCSI 1 Gb/sec). This problem with 500 GB VMs only.

Processing during backup and replication:
0 - 100 GB: speed is more 100 MB/sec
after 100GB: speed decreases (40-50 MB/sec)
after 200 GB: more decreases
and more.
After 300 GB speed is about 18 MB/sec.

If I turn CBT to off, process is faster (!!!). Speed starts decrease after 200 GB.

Can developers test backup on 500 GB VMs?

[Gostev]

Hello Andrew, we test on all backups sizes... it is not uncommon for our customers to have VMs of size over 1TB, and the backup just flies on those. Here's some real numbers gathered from this very forum, there are some very large VMs mentioned in the document.

I think it would be best if you open a support case and let our technical staff take a look at this in live environment, as this is clearly environment-specific issue and unexpected.

[solstice]

Interestingly we noticed the same thing on our network.
Our setup is Dell R710 dual quadcore (2.66) 24Gig Ram, with a MD3000i behind it. The backup server (specific for veeam) is connected to the R710 and MD3000i via a Cisco 4948 iscsi capable core switch.

Performance wise we noticed that the initial 100Gb we get about 62MB/sec .. after 100Gb it seems to fail over to network mode and then only get slowly decreasing to about 15MB/sec. We run backups of VM's in the size of 1.9Tb.

[Gostev]

Actually, failover between modes is not possible while VM is being processed. The processing mode is selected before VM processing starts.

[solstice]

hmm still running:

8 of 8 files processed
Total VM size: 1.90 TB
Processed size: 1.06 TB
Processing rate: 10 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 28/05/2010 10:08:42 PM
Time remaining: 00:42:53

Backing up file "[Data1] Server/Server_1-flat.vmdk"

Backing up file "[VM] Server/Server-flat.vmdk"
Unable to establish direct connection to the shared storage (SAN).
Please ensure that:
- HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly.
- SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server.
- Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation).

Direct SAN connection is not available, failing over to network mode...

Thus it seems to be doing fail-overs when performing multiple disk backup, if it can't access one disk in SAN mode. speed has become atrocious!

[Vitaliy S.]

Hello Sascha,

Could you tell us if you have all the disks located on the same LUN or you have multiple LUNs? If so, do you have all the LUNs correctly presented to your Veeam backup server?

Thanks!