Load levels - What is ideal?

[sbbots]

I am curious what the ideal load levels should look like for a backup job. There is always going to be a bottleneck somewhere, but if you had almost perfect synchronicity should all of the levels be at or near 0%? Or should they all be at equal levels? Or do the load levels simply indicate the performance threshold levels of each component (eg "65%" means the backup is only using 65% of that component's total bandwidth-processing power-writespeed available)?



Thank you for any insight.

[Gostev]

Hi Matt, please read the sticky FAQ topic. Bottleneck analysis logic is explained there in great details, including what these numbers indicate. Thanks!

[sbbots]

Fair enough, but the sticky only explains what the numbers mean; It does not explain what the ideal efficiency would look like. Seems like 50% across the board (average over the course of an entire job) would mean that everything is running as close to perfection as one can get.

Anyway, I will learn to use the search button

[tsightler]

I always think of it like a simplified view of a production line. Imagine if the bottleneck numbers represent the utilization of various portions of the line. The "Source" represent the receiving department where raw materials come into the plant, the "Proxy" represents the production equipment pumping out the finished goods, the "Network" represents the conveyor that transports those goods to the shipping department and the "Target" is the shipping department loading up trucks to send out those goods.

So, what's the ideal efficiency in a factory? The holy grail is to keep every section of the plant running at 100% efficiency, so that's really the same answer to your theoretical "ideal efficiency" of a Veeam B&R setup. In this theoretical perfectly balanced setup you'd keep the source 100% busy, which would be the perfect amount of data to keep the proxy 100% busy (i.e. no waster CPU), which would generate the exact traffic to keep the network 100% busy (no wasted bandwidth) and that would be the exact workflow to keep the target disk 100% busy (no wasted I/O). OK, perhaps in a ideal factory you're not quite at 100%, perhaps only the upper 90's, so that you can handle ebb and flow of normal load, but having all numbers high is still generally the ideal to strive for.

What you definitely don't want is any specific number to be way out of whack, that's when you start really having a bottleneck. If the conveyor (network) is 99% busy, but your shipping department is only 50% busy, it doesn't really matter what you do in the receiving and productions sections as the conveyor will back up the work simply leaving to more wasted resources. If you increase the capacity of the conveyor, then the utilization of the shipping department should also go up, but the utilization of production and receiving lines should also increase because previously they had to shut down waiting on the conveyor. If you increase the capacity of the conveyor enough, eventually one of the other 3 components will hit 100% and it will be the bottleneck, so then the process starts over again.

Of course, this isn't a perfect analogy and I've taken a few liberties with the explanations, but I think it's good enough to get the point across. There's probably no real-world scenario that would likely approach this perfect efficiency, but based on the numbers you've posted it seems like your setup is in pretty good shape. You have some headroom in every stage of the process and good utilization throughout. Unlike most production lines, the cost of having a little headroom isn't particularly high so not having perfect 100% utilization of every step is a good thing.

[dellock6]

Really nice analogy here, well done Tom

Luca.

[Gostev]

This made me smile. I still remember some crazy internal forums thread a few years ago with our engineers posting YouTube videos of all sort of wicked factory conveyors in attempt to ruin this analogy I suggested

This was one of them, showing network packet loss
http://www.youtube.com/watch?v=8NPzLBSBzPI