Comprehensive data protection for all workloads
Post Reply
alex1002
Enthusiast
Posts: 25
Liked: 1 time
Joined: Jan 27, 2015 6:17 pm
Full Name: Alex
Contact:

Repo dedupe question

Post by alex1002 »

Good day,
I am setting up a Windows 2016 test repo. I want to setup dedupe for it. But there's different options. General files, virtualized backup storage etc. What's the best for veeam.
Thank you
david.buchanan
Service Provider
Posts: 42
Liked: 8 times
Joined: Jun 02, 2015 12:44 am
Full Name: David
Contact:

Re: Repo dedupe question

Post by david.buchanan »

Hi Alex,

Normally you'd just select "General Files". The virtualized backup option simply as additional extensions to the exclusion list none of which are Veeam extensions (Excluding .tmp of course :) ).

By default .ebd and .jars files are excluded. The Virtualized option adds a bunch more, bin,vsv,slp,xml,tmp, etc. This is the only difference between the two that I'm aware of.

You can find out more about Dedup in Server 2016 here if it helps: https://blogs.technet.microsoft.com/aus ... available/

I'd recommend you exclude a few additional items manually such as vbm extensions as I've had cases where dedup has caused issues with these files which resulted in having to run a full backup.

I'd also suggest you don't enable "Partial files" and "In use" files being dedup'd. These are not on by default but just in case you do.

Thanks,
David.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Repo dedupe question

Post by Mike Resseler »

Hi Alex,

Windows server 2016 dedupe actually brings lots of major enhancements to deduplication and really becomes interesting... There are still limitations compared to hardware dedupe boxes but that is expected :-)

Anyway, I would chose backup storage (I still need to start testing the moment v9.5 releases...) but in the end, it really doesn't matter what you choose.
General files gives you these options:
- Background optimization
- Default optimization policy: Minimum file age = 3 days, Optimize in-use files = No, Optimize partial files = No
Backup option gives you this:
- Priority optimization
- Default optimization policy: Minimum file age = 0 days, Optimize in-use files = Yes, Optimize partial files = No
- "Under-the-hood" tweaks for interop with DPM/DPM-like solutions

The only one I personally would be scared off is that "under-the-hood" tweaking as I have no idea what it does for the moment. But, if you take the general setting, then you can easily change all of these options by using PowerShell or mostly in the UI also.

So you can start with the general files, and then tweak it to the same settings as backups (without those under the hood tweaks :-))

Hope it sheds some light
Mike
alex1002
Enthusiast
Posts: 25
Liked: 1 time
Joined: Jan 27, 2015 6:17 pm
Full Name: Alex
Contact:

Re: Repo dedupe question

Post by alex1002 »

Thank you for advice
It seems I was able to save 2TB so far.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Repo dedupe question

Post by Mike Resseler »

Alex,

I can't say if it is much or not... 2TB out of a 100 would be low, 2 TB out of 4 seems better so... ;-)
alex1002
Enthusiast
Posts: 25
Liked: 1 time
Joined: Jan 27, 2015 6:17 pm
Full Name: Alex
Contact:

Re: Repo dedupe question

Post by alex1002 »

Sorry I got 14tb

Sent from my Nexus 6P using Tapatalk
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Repo dedupe question

Post by Mike Resseler »

Alex,

That seems rather low... But it might increase the next days. With windows dedupe it can take some time in the beginning. Keep an eye on it and look if it doesn't give you issues if it doesn't succeed in time to get everything deduped. If that happens, change the minimum file age day to a higher number and then gradually lower it until zero

Cheers
Mike
alex1002
Enthusiast
Posts: 25
Liked: 1 time
Joined: Jan 27, 2015 6:17 pm
Full Name: Alex
Contact:

Re: Repo dedupe question

Post by alex1002 »

Mike Resseler wrote:Alex,

That seems rather low... But it might increase the next days. With windows dedupe it can take some time in the beginning. Keep an eye on it and look if it doesn't give you issues if it doesn't succeed in time to get everything deduped. If that happens, change the minimum file age day to a higher number and then gradually lower it until zero

Cheers
Mike
Thanks for the advise. I'll keep you posted. Just curious how does windows still manage to save space. I taught.veeam did it's own compression and dedupe magic.

Sent from my Nexus 6P using Tapatalk
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Repo dedupe question

Post by mkretzer »

We did a test with backup settings to normal (not optimized for Dedup) with Per-VM backup Files and got 24 % space saving...
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Repo dedupe question

Post by Mike Resseler »

@Alex: We do indeed have compression and dedupe but there are differences. We do this per backup file and windows per volume. This is just one example. There is also different algorithm and so on and so forth :-)

@mkretzer: What are your settings for Dedup if I may ask? (See above for the settings I'm looking for :-))

Thanks
Mike
mkretzer
Veeam Legend
Posts: 1140
Liked: 387 times
Joined: Dec 17, 2015 7:17 am
Contact:

Re: Repo dedupe question

Post by mkretzer »

We used the virtualized backup setting in TP5. Because that's what veeam backups are I think? :-)

Gesendet von meinem Pixel XL mit Tapatalk
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Repo dedupe question

Post by Mike Resseler »

True...

But one of the things is that it has some under the hood tweaks specifically for DPM... And since I don't know what those are... I basically use the other settings and apply them manually :-)
Post Reply

Who is online

Users browsing this forum: justin.hendren and 212 guests