Slow Veeam Backups from one ESXi host

[graycj]

Hi All,

I recently upgraded two VMware ESXi servers in a customer environment to the latest patches for 5.1 so they are both currenty running 5.1.0 1743533. They both have local storage and we backup to a Windows 2008 server running Veeam 7 patch 4.

After the upgrade vms on one of the ESXi servers backups up fine, the usual backup rate is approx 70-80MB/s, however the other ESXi server I've not got a good backup from yet, the rate of backup is 77KB/s!

I've created new Veeam jobs and tried backing up the vm's individually to see if it is a vm issue, but it seems specific to that ESXi server.

My questions are: How can I analyse the veeam logs in more depth to see the differences between the two jobs in action? Do you think a restart of the Management agents on the ESXi server with the slow backups will resolve the issue and if so will it have an impact on the running vms?

Thanks for your help,
Carl

[foggy]

Carl, could you please tell whether Veeam B&R is installed on a standalone physical server? Do you have Veeam B&R proxy servers deployed on the hosts? Please open session log for both jobs (right-click the job and click Statistics) and check what operations do take most of the time. What are the bottleneck stats for the slower job and what transport mode is used to retrieve VMs data? Are you saying that it used to perform decently and started to crawl right after the upgrade?

[graycj]

Hi foggy, that was quick!

The veeam server is a standalone physical server.
We don't have proxy servers deployed on the hosts.
The operation that takes the most time is the Hard Disk backup [CBT], this is the same for the Esxi host which backs up ok, and the one that doesn't. In both cases as well the bottle neck is the source.
On the server that backs up ok, the read rate is between 70 - 80MB/s, on the server that is having issues, the read rate is 74KB/s!
The original backup jobs were both working at 70-80Mb/s before i did the upgrade, it's only been since the upgrade on on ESXi host that that problem occurs, the other ESXi host is backing up without an issue.

Thanks, Carl

[Vitaliy S.]

Hi Carl,

What about other operations in the session log for the jobs that are targeted to the host that has an issue? Can you please try to use Datastore Browser and upload/download some files to both hosts and then compare the performance rates?

Thanks!

[graycj]

Hi Vitally,
I uploaded a 576MB iso image in about 30 seconds from the vcenter/Veeam physical server, using the datastore browser, once it was there I selected it to download to back to the vcenter/Veeam physical server and it indicates it is going to take 176 minutes! The same operation to the ESXi server which is working correctly takes about 30 seconds to upload to the ESXi server, but about 15 to download back to the vcenter/Veeam server.
Thanks, Carl.

[Vitaliy S.]

Can you please check what network configuration you have in the Host -> Configuration -> Network -> Switch properties -> Network adapters settings? If these values are the same across both hosts, then try to restart management agents on the "problem" ESXi host.

[graycj]

Hi Vitaliy,

A restart of the ESXi management agents did the trick. Thanks for your help.

[Vitaliy S.]

Glad that it was a just problem with agents and not network switches that would be really hard to catch.