Strategies for full backups that run too long

VMware specific discussions

Re: Strategies for full backups that run too long

Veeam Logoby pshute » Mon Jan 23, 2017 10:22 pm

foggy wrote:Bottleneck source means that data cannot be retrieved from the storage any faster. Currently source data reader is the slowest component in the data processing chain, while other components are able to process more data, but just sitting and waiting for it. Giving them the ability to process more data will result in overall backup performance increase (and bottleneck will probably shift to another component).

OK, that makes sense - it's reading from source 99% of the time, and is always making other components wait. But how then did allowing more concurrent tasks manage to get more data from the host? How can the host send data more quickly just by doing multiple streams?

You can look up previous jobs stats in the sessions History.

I can see duration, processing rate, processed, read and transferred for all sessions, but they don't tell me everything that's on the graphs. I'm interested to see what maximum transfer rate we achieved, and whether there were periods of low speeds.

The processing rate seems a bit deceptive. It's the amount of data processed divided by the job duration. But swap file and deleted blocks don't actually get read, making the rate higher than the actual read rate. But all the statistics to do with actual read rates are unavailable for all but the most recent job.
pshute
Expert
 
Posts: 118
Liked: 8 times
Joined: Mon Nov 23, 2015 10:56 pm
Full Name: Peter Shute

Re: Strategies for full backups that run too long

Veeam Logoby foggy » Tue Jan 24, 2017 12:56 pm 1 person likes this post

pshute wrote:But how then did allowing more concurrent tasks manage to get more data from the host? How can the host send data more quickly just by doing multiple streams?

Storage systems often perform better with multiple parallel streams instead of a single one.

pshute wrote:The processing rate seems a bit deceptive. It's the amount of data processed divided by the job duration. But swap file and deleted blocks don't actually get read, making the rate higher than the actual read rate. But all the statistics to do with actual read rates are unavailable for all but the most recent job.

Processing rate is calculated off the actually read data. But overall, I can see your point here.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Strategies for full backups that run too long

Veeam Logoby pshute » Fri Jan 27, 2017 4:22 am 2 people like this post

Now that we've increased the number of concurrent tasks and rearranged the order of the VMs so that the biggest ones start backing up first, and also enabling deleted block skipping, our full backups are done in 8h20, down from 17 hours. We now have a few hours of time left in the night we can expand into. No need to try installing a proxy on the host for now. Thanks for your help with this.
pshute
Expert
 
Posts: 118
Liked: 8 times
Joined: Mon Nov 23, 2015 10:56 pm
Full Name: Peter Shute

Re: Strategies for full backups that run too long

Veeam Logoby pshute » Tue Jan 31, 2017 9:33 pm

A possibly unrelated question - would enabling Per-VM Backups improve the speed where the bottleneck is Source? I'm think no, but I'm interested in enabling it anyway, so I'd like to know if there's an additional advantage.

The main reason I'm tempted to enable it is that our repository is short on space to do restores from tape. If I change to Per-VM, I will be able to restore just a single machine's backup without having to have enough room for the whole backup? Are there any disadvantages to splitting the backups up, apart from having way more files in the repository?
pshute
Expert
 
Posts: 118
Liked: 8 times
Joined: Mon Nov 23, 2015 10:56 pm
Full Name: Peter Shute

Previous

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 10 guests