low restore speed

[casemaster]

Hello all,

I have a really weird issue with Veeam B&R (version 8, Update 3)...

But first to the current structure: We have a VMware standalone host that hosts the Veeam VM. As a backup repository we use a HP StoreOnce 4500. We are backing up a VMware environment (5.5) with about 260 VM clients and have HP StoreVirtual SAN nodes (iscsi, 10 GBit). When we perform backups we gain a speed between 20 - 100 mb/s. As backup proxies we use 2 VM clients.

But now we come to the issue: When performing a Full VM restore (quick rollback) it takes ages to finish. The speed is about 300 kb/s in the best case. Just to restore the little .vmx, .vmxf and .nvram files at the beginning of the restore take about 20 minutes!!! And these files are about 11 kb... So the estimated time for a restore of a 30 GB VM is about 24 hours... that's no quick rollback...

But, if I switch to a proxy that is not part of the vsphere environment (e.g. the Veeam Server) then the speed is better. Still not as good as it should be, but around 1 MB/s.

Do you have any input, of how I can identify the bottleneck? I performed test restores some months ago and there we had 10-12 MB/s what was quite acceptable.

Thanks in advance

PS: File level restore works fine and fast...

[alanbolte]

Restore of the VM configuration files in a Full VM Restore job is currently not well optimized for slow or deduplicating storage. For a large backup file, several GB may need to be processed to restore this 11kb VMX file. This speed is not representative of the restore speed for the VM disks.

Generally, deduplicating storage (the StoreOnce) will offer poor restore performance relative to non-dedupe storage. If you have enough space on the backup server or proxy for a single full backup file, you could try restoring from that for comparison. Also, are you using active full backups, or one of the various synthetic modes?

It sounds like you already partially identified a bottleneck by changing proxies, you just have to identify whether using the proxy was faster due to better speeds from the StoreOnce to that proxy, or better speeds from the proxy to your ESXi host, (may be related to transport mode instead of bandwidth), or if resource usage (e.g. CPU/memory) is the issue.

As to file level restore, when you say it is fast, how fast? It shouldn't be any faster than you can restore a VMDK file to the same location...

[casemaster]

Hello all,

just want to inform that the issue is solved. It was no issue with the storage or VEEAM. Just a simple switch the StoreOnce was connected to was the cause that slowed down the transfer speed.

Thanks again for your input!