Repo dedupe question

Availability for the Always-On Enterprise

Repo dedupe question

Veeam Logoby alex1002 » Mon Oct 31, 2016 12:01 am

Good day,
I am setting up a Windows 2016 test repo. I want to setup dedupe for it. But there's different options. General files, virtualized backup storage etc. What's the best for veeam.
Thank you
alex1002
Enthusiast
 
Posts: 25
Liked: 1 time
Joined: Tue Jan 27, 2015 6:17 pm
Full Name: Alex

Re: Repo dedupe question

Veeam Logoby david.buchanan » Mon Oct 31, 2016 6:18 am

Hi Alex,

Normally you'd just select "General Files". The virtualized backup option simply as additional extensions to the exclusion list none of which are Veeam extensions (Excluding .tmp of course :) ).

By default .ebd and .jars files are excluded. The Virtualized option adds a bunch more, bin,vsv,slp,xml,tmp, etc. This is the only difference between the two that I'm aware of.

You can find out more about Dedup in Server 2016 here if it helps: https://blogs.technet.microsoft.com/aus ... available/

I'd recommend you exclude a few additional items manually such as vbm extensions as I've had cases where dedup has caused issues with these files which resulted in having to run a full backup.

I'd also suggest you don't enable "Partial files" and "In use" files being dedup'd. These are not on by default but just in case you do.

Thanks,
David.
david.buchanan
Enthusiast
 
Posts: 41
Liked: 8 times
Joined: Tue Jun 02, 2015 12:44 am
Full Name: David

Re: Repo dedupe question

Veeam Logoby Mike Resseler » Mon Oct 31, 2016 7:43 am

Hi Alex,

Windows server 2016 dedupe actually brings lots of major enhancements to deduplication and really becomes interesting... There are still limitations compared to hardware dedupe boxes but that is expected :-)

Anyway, I would chose backup storage (I still need to start testing the moment v9.5 releases...) but in the end, it really doesn't matter what you choose.
General files gives you these options:
- Background optimization
- Default optimization policy: Minimum file age = 3 days, Optimize in-use files = No, Optimize partial files = No
Backup option gives you this:
- Priority optimization
- Default optimization policy: Minimum file age = 0 days, Optimize in-use files = Yes, Optimize partial files = No
- "Under-the-hood" tweaks for interop with DPM/DPM-like solutions

The only one I personally would be scared off is that "under-the-hood" tweaking as I have no idea what it does for the moment. But, if you take the general setting, then you can easily change all of these options by using PowerShell or mostly in the UI also.

So you can start with the general files, and then tweak it to the same settings as backups (without those under the hood tweaks :-))

Hope it sheds some light
Mike
Mike Resseler
Veeam Software
 
Posts: 3340
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Repo dedupe question

Veeam Logoby alex1002 » Mon Oct 31, 2016 2:54 pm

Thank you for advice
It seems I was able to save 2TB so far.
alex1002
Enthusiast
 
Posts: 25
Liked: 1 time
Joined: Tue Jan 27, 2015 6:17 pm
Full Name: Alex

Re: Repo dedupe question

Veeam Logoby Mike Resseler » Mon Oct 31, 2016 2:56 pm

Alex,

I can't say if it is much or not... 2TB out of a 100 would be low, 2 TB out of 4 seems better so... ;-)
Mike Resseler
Veeam Software
 
Posts: 3340
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Repo dedupe question

Veeam Logoby alex1002 » Tue Nov 01, 2016 2:45 am

Sorry I got 14tb

Sent from my Nexus 6P using Tapatalk
alex1002
Enthusiast
 
Posts: 25
Liked: 1 time
Joined: Tue Jan 27, 2015 6:17 pm
Full Name: Alex

Re: Repo dedupe question

Veeam Logoby Mike Resseler » Tue Nov 01, 2016 4:38 am

Alex,

That seems rather low... But it might increase the next days. With windows dedupe it can take some time in the beginning. Keep an eye on it and look if it doesn't give you issues if it doesn't succeed in time to get everything deduped. If that happens, change the minimum file age day to a higher number and then gradually lower it until zero

Cheers
Mike
Mike Resseler
Veeam Software
 
Posts: 3340
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Repo dedupe question

Veeam Logoby alex1002 » Wed Nov 02, 2016 2:10 am

Mike Resseler wrote:Alex,

That seems rather low... But it might increase the next days. With windows dedupe it can take some time in the beginning. Keep an eye on it and look if it doesn't give you issues if it doesn't succeed in time to get everything deduped. If that happens, change the minimum file age day to a higher number and then gradually lower it until zero

Cheers
Mike

Thanks for the advise. I'll keep you posted. Just curious how does windows still manage to save space. I taught.veeam did it's own compression and dedupe magic.

Sent from my Nexus 6P using Tapatalk
alex1002
Enthusiast
 
Posts: 25
Liked: 1 time
Joined: Tue Jan 27, 2015 6:17 pm
Full Name: Alex

Re: Repo dedupe question

Veeam Logoby mkretzer » Wed Nov 02, 2016 5:25 am

We did a test with backup settings to normal (not optimized for Dedup) with Per-VM backup Files and got 24 % space saving...
mkretzer
Expert
 
Posts: 337
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Repo dedupe question

Veeam Logoby Mike Resseler » Wed Nov 02, 2016 6:12 am

@Alex: We do indeed have compression and dedupe but there are differences. We do this per backup file and windows per volume. This is just one example. There is also different algorithm and so on and so forth :-)

@mkretzer: What are your settings for Dedup if I may ask? (See above for the settings I'm looking for :-))

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 3340
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Repo dedupe question

Veeam Logoby mkretzer » Wed Nov 02, 2016 7:25 pm

We used the virtualized backup setting in TP5. Because that's what veeam backups are I think? :-)

Gesendet von meinem Pixel XL mit Tapatalk
mkretzer
Expert
 
Posts: 337
Liked: 74 times
Joined: Thu Dec 17, 2015 7:17 am

Re: Repo dedupe question

Veeam Logoby Mike Resseler » Wed Nov 02, 2016 7:27 pm

True...

But one of the things is that it has some under the hood tweaks specifically for DPM... And since I don't know what those are... I basically use the other settings and apply them manually :-)
Mike Resseler
Veeam Software
 
Posts: 3340
Liked: 379 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Exabot [Bot], Google Feedfetcher, Mandietha, spiritie and 65 guests