Backup of VM (suddenly) very slow (twice the duration)

[DennisV]

Veeam backup server is running in VM.
Source VMDK's are on local storage.
Backup destination is NAS.

According to Veeam Statistics :
Total size : 235GB
Avg. perf. rate. : 7 MB/s
Start : 0100
End : 1100
Durarion 10 hours !

Actual vbk size : 189 GB
Actual vrb size : approx. 3,5 GB

This VM used to backup in less than half the time (4 hour) with a perf. rate of 18MB/s

When I copy data from the backup VM to the NAS things go a lot faster , around 20MB/s.
Using WinSCP I have done some speed measurements.
From the ESX host to the NAS (via the backup VM server), around 20MB/s.
From the NAS to the ESX host (via the backup VM server), around 5MB/s.
Stange thing I notices the huge drops and variations in write speed to the ESX host.
12MB/s, few KB/s, 18MB/s, few KB/s etc. making the avarage speed very low.

Could this be the problem of the slow backup ?

Edit :

ESX 4.0 (no update)
Veeam 3.1.1

[Gostev]

In most cases, such issues are caused by some switch or NIC port on Veeam Backup server, ESX or destination server or in-between them failing over to 100Mbit from 1Gb.

[DennisV]

I've tried setting the NICs to Fixed 1000 Full, no changes.
Read speeds from both host seem ok, but write is a lot slower and fluctuating.
hdparm confirm read speeds are acceptable (host 1 35MB/s, host 2 130MB/s)

I tried a dd write test on the hosts....seems consistent with the low speeds I get.
This means the actual write throughput to the disks is more than terrible.
5MB/s for host 1 and 7MB/s for host 2....ouch....

How much is Veeam affected by these slow write speeds of the hosts ?

[Gostev]

Veeam Backup only reads from the hosts, so I don't believe host write speed could be an issue.

[DennisV]

The read speed from the hosts to both the backup server and to the NAS are much better (20MB/s+).
So how come Veeam Backup is much slower.

These slow speeds are only with the backup server running in the VM.
I've monitored the cpu and memory of this VM during the backup, both have enough free resources.
Any other suggestions ?

( P.S. yes I will try to upgrade this customer to 4.1 asap )

[Gostev]

Veeam Backup "processing rate" is not the same as ESX read speed, as this counter averages against whole time it took to backup. This includes snapshot operations, vCenter interactions, small config files processing - which may all take significant time thus affecting the counter. So obviously you cannot compare some random large file copy speed, and this counter directly.

Unfortunately it is pretty impossible to troubleshoot most technical issues remotely, without webex and full logs, this is exactly why we suggest that you contact support directly for any technicall issues, instead of posting about it on forums.