Understanding job statistics with WAN Acceleration

[christiankelly]

I’m trying to understand the job statistics for a “copy-job” with WAN acceleration enabled.

I setup a copy-job of a busy server. The daily incremental backup is around 180-200 GB and once things baselined I was getting a job that “processed” around 350GB and transferred between 35GB to 45GB which was great. What I didn’t understand was, the statistics were showing very little as “Obtained from WAN Acceleration cache”.



You can see from the above graphic that from the 3 disks it shows about 3.5GB total as “obtained from WAN Acceleration”. That led me to believe that if I took the acceleration out, it would only be on average around 3.5GB in extra transferred data.



I changed the job to copy direct and let it run normally to see what would happen. This time as you can see from the graphic, it read about the same amount (310GB) however the transferred was 172GB so clearly the WAN accelerator is doing much more than just the 3.5GB it was able to use from cache.

I didn’t see anything else in the statistics that was showing the savings in bandwidth or how it was being optimized. So I guess my question is how can one tell how much would be copied without the WAN accelerator inline?



In this last graphic you can see the destination of the job in the remote site. The 203GB file is from the first job which had a transfer of 41GB and the 180GB file is from the second job.

As you can imagine, I’m very happy with the reduction in bandwidth, but as a second part of this question, it still takes a very long time to run the job even though it’s only copying 41GB. The “bottleneck” shows between “Source WAN” and “Target WAN”. What would help speed things up? I checked CPU and memory and both seemed ok, are we talking IOPS here? I would like to get this job to run in under 20 hours if possible and I should be able to transfer 40GB in about 5 hours with the bandwidth I have, so I guess it’s all going to be about optimizing the accelerators?

[foggy]

I didn’t see anything else in the statistics that was showing the savings in bandwidth or how it was being optimized.

Reduction comes from analyzing previous restore points (those that are already stored on the target backup repository) also. Those blocks that are found there, are eliminated from the data transfer. I recommend to review the corresponding user guide section (p.120) for more details on how data deduplication works with WAN accelerated jobs.

Another point to mention is that WAN accelerators use much smaller block size compared to typical backup or backup copy job in direct mode, which also results in data reduction.

As you can imagine, I’m very happy with the reduction in bandwidth, but as a second part of this question, it still takes a very long time to run the job even though it’s only copying 41GB. The “bottleneck” shows between “Source WAN” and “Target WAN”. What would help speed things up? I checked CPU and memory and both seemed ok, are we talking IOPS here?

Yes, exactly. Since WAN accelerator is effectively trading disk I/O for WAN bandwidth savings, disk I/O performance on the target repository is in most cases the primary bottleneck in the WAN accelerator process. Among tips for performance optimization in this case is placing global cache on a separate disk from the repository, probably SSD disk that provides fast random I/O.

[christiankelly]

So is the only way to see how much data would have transferred over the wire without acceleration to check the filesize on other side and compare it with the “transferred” number that’s reported? It's odd that Veeam wouldn't show the full savings "over the wire" on the statistics page.