Deduplication Consideration

[futureweb]

Hi,

I read your FAQ about DEDUP and understand, that you dedup on a per "job" base.
I do wonder now, if it would be a good idea to use Windows 2012 Dedup in our case.
I read a best practise that was for Veeam 6.5.

If we go this road a question:

1) We usually do forever incremental. If a incremental is reinjected into a VBK, are only blocks changed, that have been modified by the next incremental?
I ask, because DEDUP on Server 2012 take quite ressources, especially if a VBK is several TB big. Also the garbage collection once a week takes ressources. It wouldn't make sence, if it would dedup a VBK and would have to Dedup it again every day,
and garbage collection would need to work hard each week too.

On the other side, if only changes within the big file are injected, MS Dedup might only need to dedup the changes, and even the VBK is several TB big, changes per day are less than 100 GB.

Do you have any experience or recommendations about this?

We only archive backups on disk for the last 3 weeks, every longer retention goes on tape.

Thanks
Patrick

[veremin]

Windows 2012 deduplication doesn't work well for forever backup modes. You'd achieve better results with a mode using periodic fulls. Other recommendations can be found here. Thanks.

[dellock6]

Hi Patrick,
windows dedup is file-based, so an injected/updated file as it's in forever incremental will be processed from scratch. As said, periodic fulls are the best options, and I may suggest to have a look at the upcoming Windows 2016 dedupe engine, it's moving from single-thread to multi-thread and remove the limit of 2TB volumes up to 64TB. Still the single file needs to be smaller than 1TB, but by using the new upcoming per-vm backup chains in Veeam V9 this limit can be worked around for the vast majority of VMs. The Veeam dedupe you looseby going to per-vm chains is re-gained with the windows dedupe.

Luca

[futureweb]

Ok thanks for your information, that helps, but we will wait and look into this again once V9 and Server 2016 is both on the road.

Patrick