Improving backup speed and time window / duration

Availability for the Always-On Enterprise

Improving backup speed and time window / duration

Veeam Logoby albertwt » Sun Jul 24, 2016 11:12 am

People,

I'm having a problem with the large VM backup window that is getting longer lately.

The VMs:
Exchange Server Mailbox servers ~ 8 TB+ each VM took 17 hrs to complete.
SQL Servers ~ 2 TB each VM took 7-8 hrs to complete.
File Servers NAS ~ 5-6 TB took 3-4 hrs to complete.

Backup Repository:
LUN 1 - RAID 50 [HP D2700 25x 450GB 6G SAS 10K SFF Dual Port HDD 11.2TB]
LUN 2 - RAID 6 ADG [HP D2700 25x 450GB 6G SAS 10K SFF Dual Port HDD 11.2TB]
LUN 3, 4, 5 and 6 - RAID 5 [HP D2600 12x 3TB 6G SAS 7.2K LFF Dual port MDL HDD 36TB]

From the VBR console, the bottleneck is the Target.

So can I convert them all from Reverse Incremental into Forward Forever incremental ?

Thank in advance.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby vmniels » Sun Jul 24, 2016 11:40 am 1 person likes this post

That would be one option. Reverse IO requires quite some IO and RAID5 and RAID6 may not be the best.

If you switch from the reverse incremental method to the forward incremental method, VBR will create a set of incremental backups next to the reverse incremental chain. The full backup in the reverse incremental chain will be used as a starting point for produced increments.

In regards to the time, are these timeframes about an incremental or full backup ?
VCP-DCV
Veeam Certified Engineer
http://foonet.be
vmniels
Veeam Software
 
Posts: 1507
Liked: 325 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: Improving backup speed and time window / duration

Veeam Logoby albertwt » Sun Jul 24, 2016 10:18 pm

Thanks Niels,

This is in regards to the incremental backup daily.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby Vitaliy S. » Mon Jul 25, 2016 10:55 am 1 person likes this post

As another possible enhancement you can consider switching to RAID 10 in any new similar deployment, should also give you a bit more IOPs performance.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Improving backup speed and time window / duration

Veeam Logoby albertwt » Mon Jul 25, 2016 2:03 pm

Hi Vitaliy,
I can't switch it to raid 10 because the disk space is running very low, hence I create raid 5 to get more disk capacity.

Does changing from reverse incremental to forward incremental can trigger full backup on the next backup schedule ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby v.Eremin » Mon Jul 25, 2016 2:10 pm 1 person likes this post

Does changing from reverse incremental to forward incremental can trigger full backup on the next backup schedule ?

The latest restore point in that case would be already a full backup, so no new full backup cycle should take place. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

[MERGED] Backing up Exchange Server taking 13+ hours to fini

Veeam Logoby albertwt » Mon Oct 17, 2016 6:57 am

Hi All,

I've got large Exchange Server (Mailbox server) VMs that is backed up by Veeam daily, but the problem is the job taking long to finish around 13+ hours

VM size total is 9 TB+ according to VBR
there are 7 VMDK in the VM that is 1.89 TB in size which hold large Exchange mailbox .edb file per drive.

Job settings:
Image

Bottleneck is on the: Target - Processing rate is 15 - 20 MB per second.

The target backup physical repo server is in the same data center same rack with the Exchange Server vSphere environment.
Image

The RAID 5 is consists of 12 x 3 TB SAS HDD.


Any kind of tips and suggestion would be greatly appreciated.

Thanks.

I'm using Veeam Backup & Replication 9.0.0.1715
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Backing up Exchange Server taking 13+ hours to finish

Veeam Logoby ACC » Mon Oct 17, 2016 8:24 am 1 person likes this post

Based on the information provided, the cause of the low performance is likely the high random IO workload. Reverse incremental is more disk space efficient, but it does come with a penalty that the workload will consist of primarily random IOs. This kind of workload on a raid 5 set (or 6 for that matter) is tough on the performance. Raid 5 is, to put it mildly, not great in handling random write IOs. To improve the performance, switch the job to forward incremental to get more sequential IO workload and/or use raid 10 which has a lesser penalty on dealing with random write IOs.
ACC
Influencer
 
Posts: 24
Liked: 5 times
Joined: Wed Apr 29, 2015 10:38 am
Full Name: ACC

Re: Backing up Exchange Server taking 13+ hours to finish

Veeam Logoby albertwt » Mon Oct 17, 2016 12:29 pm

Hi Acc,

Thanks for the suggestion.
I need to maximize the disk space or the capacity hence I selected RAID 5 in my DAE configuration.

If I change the backup job from Reverse Incremental to Forward Incremental forever, does it going to consume more disk space ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby foggy » Mon Oct 17, 2016 12:31 pm 2 people like this post

You need to switch to simple forward incremental, since forever forward will still require random I/O during transformation. Yes, you would need more space with forward incremental mode.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Improving backup speed and time window / duration

Veeam Logoby albertwt » Mon Oct 17, 2016 12:33 pm

Alex,
Is there any calculator or estimator of how much more disk space is needed for the Forever forward incremental ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby foggy » Mon Oct 17, 2016 12:36 pm 1 person likes this post

Forever forward incremental doesn't require more space, however, forward incremental does. You can perform space estimation here.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Improving backup speed and time window / duration

Veeam Logoby albertwt » Mon Oct 17, 2016 12:43 pm

Alex,

So what's the different on those two mode ?

Image

My understanding is:

Forever forward incremental mode: Incremental (Recommended) selected, Create Synthetic Full Backups periodically option unchecked and Transform previous backup chains into rollbackoption unchecked. Hence disk space usage remains the same.

Forward incremental mode: Incremental (Recommended) selected, Create Synthetic Full Backups periodically option checked and Transform previous backup chains into rollbackoption checked. Additional disk space usage needed.

Is that correct ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Improving backup speed and time window / duration

Veeam Logoby foggy » Mon Oct 17, 2016 12:56 pm 2 people like this post

Please review the corresponding user guide section.
foggy
Veeam Software
 
Posts: 14742
Liked: 1080 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Improving backup speed and time window / duration

Veeam Logoby sanjaykrk » Mon Oct 17, 2016 6:45 pm 1 person likes this post

Albertwt,
As per my understanding, Your description of two modes is correct.

In forever forward incremental mode, after first initial full backup, you will only be writing incrementals (forever).
But, once you exceed the number of restore points based on your retention policy, Veeam will merge this 'First Full backup' with 'next increment' to prepare updated Full backup to stay within the number of restore point allowed. This may involve random I/O and this merge operation can become frequent (may be somebody can confirm if my understanding is correct).
Advantage: disk space use is on the lower side (One full backup image +incrementals).

In Forward incremental, you start with a full backup, followed by incrementals and periodic Synthetic or Active Full backup.
Depdending on your retention policy, disk space utilization should stablize after a few cycles.
This should be the fastest in terms of backup window. But depending on your retention policy/restore points/frequency of full backup, your disk space usage will vary.

For Reverse, disadvantage: its slow. Advantage: you most recent backup image is always 'full'.
sanjaykrk
Influencer
 
Posts: 11
Liked: 3 times
Joined: Sat Apr 09, 2016 12:12 am
Full Name: Sanjay kumar

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 17 guests