How to troubleshoot performance issues?

[cparker4486]

Hello,

Does Veeam have a guide for troubleshooting bottlenecks?

I setup some new jobs yesterday and paid attention to the Load stats this morning. This is what they are for the full backup done last night: Source 66% > Proxy 95% > Network 5% > Target 1%.

My setup is small in that I have 2 ESXi hosts (PowerEdge R710) and SAN (EQL PS4000X in RAID 6).

The total time for the job was 6 hrs 8 mins, processed 736GB, read 707GB, transferred 301GB. Job rate was 34 MB/s.

(I can provide more information if needed.)

[ZachW]

We do not have a "guide" for troubleshooting bottlenecks.. But we do explain the bottlenecks in our user guide as well a a bit of information on the forums pertaining to the bottlenecks. The first thing in your case that I would take a look at is the proxy.. It seems as though it was at 95% for this run ( which is not too great ). What are the resources of the proxy server being used by chance?

[cparker4486]

The proxy is the backup server itself. 6GB, 2 core E5520 @ 2.26GHz.

[Yuki]

2 cores is not enough. At default settings our veeam proxy/server can easily consume 8 cores on certain jobs if source and target can keep up. Looking your numbers you need to assign more CPU power to the proxy/server that does data processing.

[cparker4486]

I will upgrade the server and report back tomorrow.

[Vitaliy S.]

Also if there is no opportunity to use a more powerful server as a proxy, you may want to reduce the compression level. This will make your job run faster, but will also increase the size of your backup files.

Btw, If you have any spare VM, use it as a secondary proxy server, but don't forget to assign at least 4 vCPUs to this VM.

[cparker4486]

I gave the machine 6 vCPUs and the bottleneck is now reported as Source 15% > Proxy 12% > Network 0% > Target 98%. The Proxy number has changed a lot and the processing rate is way up but the jobs are only doing incremental backups now so it's a bit like comparing apples and oranges. The Target is pretty slow I guess. It's a Synology DS1812+ with 6 disks.

[Yuki]

yes, that sounds right. The bottleneck still should reflect correctly despite running incremental. One way you can verify this is by looking at CPU load on proxy during the backup job. I suspect it will jump around, but on average will stay around 20-50% load. Your Synology will be the bottleneck now. Depending on your backup type and amount of data - you may find yourself needing faster storage target.

[Vitaliy S.]

The Proxy number has changed a lot and the processing rate is way up but the jobs are only doing incremental backups now so it's a bit like comparing apples and oranges.

If you're running reversed incremental backup mode, then switching to forward incremental should also increase the job processing rate, as it will produce less IO load on the target storage.