Discussions specific to the VMware vSphere hypervisor
Post Reply
marius roma
Expert
Posts: 418
Liked: 3 times
Joined: Feb 01, 2012 12:04 pm
Full Name: Mario
Contact:

Reverse Incremental in deduplicating repositories

Post by marius roma » Apr 16, 2016 12:35 pm

I do know that KB 1745 (see https://www.veeam.com/kb1745) recommends to use Incremental backup for deduplicating repositories.
I changed several jobs according to the article but the overall use of space increased, maybe due to problems in the deduplicating process.
At this point I am considering going back to Reverse incremental.
Can anybody please explain the real disadvantage of using reverse Incremental in deduplicating repositories?
Regards
Marius

vmniels
Veeam Software
Posts: 2224
Liked: 484 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by vmniels » Apr 16, 2016 4:14 pm 1 person likes this post

Could you tell a bit more about your dedupe repository? Which device is behind it?

In general reverse requires quite some I/O to perform well. On the other hand if you use for example Windows 2012 dedupe it might be you aren't seeing effects as dedupe might not happen directly on the last full backup file (since with reverse incremental the last file in the backup chain is always a full).
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be

marius roma
Expert
Posts: 418
Liked: 3 times
Joined: Feb 01, 2012 12:04 pm
Full Name: Mario
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by marius roma » Apr 18, 2016 11:10 am

We use several NAS units, including a device that runs Windows Storage Server 2012 R2.
Deduplication is confidured to operate only on files older by 3 days.
Is it advisable to confngure it to operate on files older than 0 days, that means immediately as the files are written?
We nned to fond the right compromise between speed and space...
Regards
Marius

foggy
Veeam Software
Posts: 17430
Liked: 1446 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by foggy » Apr 18, 2016 11:26 am 2 people like this post

As Niels has explained above, reverse incremental might give you decent performance due to the fact that the latest backup is not deduped immediately with your current settings and synthetic activity would not require any data re-hydration. Since reverse incremental requires less space, you can run jobs this way for some time to estimate whether it gives you an acceptable speed-space trade-off.

albertwt
Expert
Posts: 636
Liked: 20 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by albertwt » Apr 18, 2016 1:19 pm 1 person likes this post

If the Veeam Backup job is already deduplicated by the setting, how can it be possible the underlying LUN is deduped by the SAN or Windows Server 2012 R2 ?
--
/* Veeam software enthusiast user & supporter ! */

marius roma
Expert
Posts: 418
Liked: 3 times
Joined: Feb 01, 2012 12:04 pm
Full Name: Mario
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by marius roma » Apr 18, 2016 1:37 pm 1 person likes this post

You can configure jobs to implement different levels of deduplication.
The problem is detecting the best compromise between speed of the job (that is influenced by proccess time in the Veeam server to deduplicate, bandwidth used to transfer more or less data, time requested by the NAS to write data and maybe deduplicating them...) and use (or waste) of space on the repository.
Everyone can find the best Solution by try end error, or ask for advise to more expert people to avoid common mistakes...
Thank you to everybody.
Marius

foggy
Veeam Software
Posts: 17430
Liked: 1446 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by foggy » Apr 18, 2016 3:14 pm 1 person likes this post

Veeam B&R uses pretty large block size (256 KB minimum), so most of the deduplication appliances are able to dedupe backup files further due to much smaller blocks (typically of variable size).

DaveWatkins
Expert
Posts: 329
Liked: 88 times
Joined: Dec 13, 2015 11:33 pm
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by DaveWatkins » Apr 18, 2016 9:06 pm 1 person likes this post

A word of warning, if you set your Windows server to dedup on 0 days, and have maintenance and defrag of backup files enabled you can very likely get your VBK file to a point that you can't write to it (I've seen this happen twice, both volumes were formatted with /L) as the maintenance and defrag move so much data around in the file it uses up all the allocation table space. Windows can fix that by doing a defrag itself but it takes a while. In my cases this happened with large files (exchange backups) but it's worth mentioning reverse incremental's and windows dedup may not be the best combination

albertwt
Expert
Posts: 636
Liked: 20 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by albertwt » Apr 18, 2016 11:22 pm

Dave,

Many thanks for sharing the experience here.
in that case we will have to wait for the proper best practice of the VBR repository implementation on Deduping data store.
--
/* Veeam software enthusiast user & supporter ! */

Delo123
Expert
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by Delo123 » Apr 19, 2016 9:13 am 1 person likes this post

Hi Albert,

maybe you can have a look at this topic,
veeam-backup-replication-f2/backup-appl ... 99-15.html
here you can also see how we have set things up, maybe it can be of help to you, however whatever you do, do not use reverse incremental with windows dedupe,
since the physical full backup file is changed during every backup windows dedupe has to start over deduping it again everytime so it will never finish. In forward incremental every file will only need to be touched/deduped once and it can take as long as it needs even if the dedupe job doesn't complete on the first run. It will continue were it was the next time it runs...

Guido

albertwt
Expert
Posts: 636
Liked: 20 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Reverse Incremental in deduplicating repositories

Post by albertwt » Apr 19, 2016 12:30 pm

Hi Guido,

Thanks for the reply, I only have Physical windows server 2012 R2 running as the Repository for the Backup & Backup Copy job. So yes, I'll read through the article before I enable the File Server Deduping feature.
--
/* Veeam software enthusiast user & supporter ! */

Post Reply

Who is online

Users browsing this forum: No registered users and 14 guests