Reverse Incremental in deduplicating repositories

VMware specific discussions

Reverse Incremental in deduplicating repositories

Veeam Logoby marius roma » Sat Apr 16, 2016 12:35 pm

I do know that KB 1745 (see https://www.veeam.com/kb1745) recommends to use Incremental backup for deduplicating repositories.
I changed several jobs according to the article but the overall use of space increased, maybe due to problems in the deduplicating process.
At this point I am considering going back to Reverse incremental.
Can anybody please explain the real disadvantage of using reverse Incremental in deduplicating repositories?
Regards
Marius
marius roma
Expert
 
Posts: 317
Liked: 3 times
Joined: Wed Feb 01, 2012 12:04 pm
Full Name: Mario

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby vmniels » Sat Apr 16, 2016 4:14 pm 1 person likes this post

Could you tell a bit more about your dedupe repository? Which device is behind it?

In general reverse requires quite some I/O to perform well. On the other hand if you use for example Windows 2012 dedupe it might be you aren't seeing effects as dedupe might not happen directly on the last full backup file (since with reverse incremental the last file in the backup chain is always a full).
VCP-DCV
Veeam Certified Engineer
http://foonet.be
vmniels
Veeam Software
 
Posts: 1505
Liked: 325 times
Joined: Mon Jul 15, 2013 11:09 am
Full Name: Niels Engelen

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby marius roma » Mon Apr 18, 2016 11:10 am

We use several NAS units, including a device that runs Windows Storage Server 2012 R2.
Deduplication is confidured to operate only on files older by 3 days.
Is it advisable to confngure it to operate on files older than 0 days, that means immediately as the files are written?
We nned to fond the right compromise between speed and space...
Regards
Marius
marius roma
Expert
 
Posts: 317
Liked: 3 times
Joined: Wed Feb 01, 2012 12:04 pm
Full Name: Mario

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby foggy » Mon Apr 18, 2016 11:26 am 2 people like this post

As Niels has explained above, reverse incremental might give you decent performance due to the fact that the latest backup is not deduped immediately with your current settings and synthetic activity would not require any data re-hydration. Since reverse incremental requires less space, you can run jobs this way for some time to estimate whether it gives you an acceptable speed-space trade-off.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby albertwt » Mon Apr 18, 2016 1:19 pm 1 person likes this post

If the Veeam Backup job is already deduplicated by the setting, how can it be possible the underlying LUN is deduped by the SAN or Windows Server 2012 R2 ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby marius roma » Mon Apr 18, 2016 1:37 pm 1 person likes this post

You can configure jobs to implement different levels of deduplication.
The problem is detecting the best compromise between speed of the job (that is influenced by proccess time in the Veeam server to deduplicate, bandwidth used to transfer more or less data, time requested by the NAS to write data and maybe deduplicating them...) and use (or waste) of space on the repository.
Everyone can find the best Solution by try end error, or ask for advise to more expert people to avoid common mistakes...
Thank you to everybody.
Marius
marius roma
Expert
 
Posts: 317
Liked: 3 times
Joined: Wed Feb 01, 2012 12:04 pm
Full Name: Mario

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby foggy » Mon Apr 18, 2016 3:14 pm 1 person likes this post

Veeam B&R uses pretty large block size (256 KB minimum), so most of the deduplication appliances are able to dedupe backup files further due to much smaller blocks (typically of variable size).
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby DaveWatkins » Mon Apr 18, 2016 9:06 pm 1 person likes this post

A word of warning, if you set your Windows server to dedup on 0 days, and have maintenance and defrag of backup files enabled you can very likely get your VBK file to a point that you can't write to it (I've seen this happen twice, both volumes were formatted with /L) as the maintenance and defrag move so much data around in the file it uses up all the allocation table space. Windows can fix that by doing a defrag itself but it takes a while. In my cases this happened with large files (exchange backups) but it's worth mentioning reverse incremental's and windows dedup may not be the best combination
DaveWatkins
Expert
 
Posts: 252
Liked: 61 times
Joined: Sun Dec 13, 2015 11:33 pm

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby albertwt » Mon Apr 18, 2016 11:22 pm

Dave,

Many thanks for sharing the experience here.
in that case we will have to wait for the proper best practice of the VBR repository implementation on Deduping data store.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby Delo123 » Tue Apr 19, 2016 9:13 am 1 person likes this post

Hi Albert,

maybe you can have a look at this topic,
https://forums.veeam.com/veeam-backup-replication-f2/backup-appliance-data-domain-vs-exagrid-vs-quantum-dxi-t31199-15.html
here you can also see how we have set things up, maybe it can be of help to you, however whatever you do, do not use reverse incremental with windows dedupe,
since the physical full backup file is changed during every backup windows dedupe has to start over deduping it again everytime so it will never finish. In forward incremental every file will only need to be touched/deduped once and it can take as long as it needs even if the dedupe job doesn't complete on the first run. It will continue were it was the next time it runs...

Guido
Delo123
Expert
 
Posts: 348
Liked: 94 times
Joined: Fri Dec 28, 2012 5:20 pm
Full Name: Guido Meijers

Re: Reverse Incremental in deduplicating repositories

Veeam Logoby albertwt » Tue Apr 19, 2016 12:30 pm

Hi Guido,

Thanks for the reply, I only have Physical windows server 2012 R2 running as the Repository for the Backup & Backup Copy job. So yes, I'll read through the article before I enable the File Server Deduping feature.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Expert
 
Posts: 607
Liked: 19 times
Joined: Thu Nov 05, 2009 12:24 pm
Location: Sydney, NSW


Return to VMware vSphere



Who is online

Users browsing this forum: Yahoo [Bot] and 14 guests