Dedupe, VM based backup chains and version 9

[DaveWatkins]

Hi

I'm wondering if anyone can answer a question about version 9 and the VM based backup chains. Currently we are using Win2012R2 dedupe on our storage repos to consolidate space. This works well but we're running up against trouble with our Exchange servers. They are about 2TB each and so end up being in a file to large for Windows to Dedupe. The great shame of that is that they would have some of the best dedupe capability since it's just identical copies of the same stores.

Many people have asked for options to split backup files but that doesn't seem to have ever happened. Hints have been dropped about how the per VM backup chains in version 9 will solve this problem but what I'd like to know is if this will help at all when a single VM is over 1TB? Per VM backup chains aren't going to help me as far as I can tell unless I can also set splitting into 800GB-1TB files unless I'm missing something about the per VM Backup chains.

Can anyone shed any light on this, or do I just have to wait and see when v9 comes out?

Thanks

[dellock6]

per-vm backup chain will store each VM in a separate backup file, both full or incremental.
So, your worst case will be now the biggest VM stored in a full backup. But remember we also have compression and deduplication, and especially compression is going to be extremely effective even for a single VM, and if the original size is 2TB, we usually se 2x size reduction thanks to it. So the final size can be around 1TB, thus a good size for Windows dedupe.
There's also Bitlooker coming, that will be able to remove deleted blocks. We didn't post additional info yet, but some of our Veeam Vanguards did, like this: https://blog.workinghardinit.work/2015/ ... cation-v9/

So, the final size with the two combined features can really be your solution.

[DaveWatkins]

Hi Luca

Thanks for the info. Currently I've been using dedupe-friendly compression (for obvious reasons), and we don't get anywhere near 2:1 reduction in size on our exchange VM's. You've got me wondering now though if it doesn't make more sense to turn up the compression to try and get he file size down so that Windows dedupe will at least try and dedupe it. It won't get anywhere near as good dedupe but for that job that's probably the way to go.

Thanks
Dave

[Delo123]

Dave,
our Exchange backups are way over 2TB, also have some Filservers which end up in a 4TB+ Backup File, actually i think nearly all our fulls are over 2TB...
Anyway, we have been running 2012R2 for years now without issues... We just needed to tweak the dedup jobs a little so they run faster, but usually all dedupe jobs are done before the next backup cycle!

PS C:\Windows\system32> get-dedupstatus

FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
53 TB 31.95 TB 138 134 J:
46.8 TB 70.15 TB 178 178 G:
55.09 TB 38.57 TB 213 213 I:
34 TB 179.24 TB 469 470 H:

[DaveWatkins]

Thats good to hear, thanks. Can you tell me what you did to tweak the jobs?

[Delo123]

Have a look at the scheduled task settings. Set priority to highest and memory to 100% (which seems te need about 2-4gb of ram per volume).
Also if you have multiple jobs, you could distribute them across multiple volumes and thus run dedupe in parallel...

Ps. I am quite sure there are some performance improvments in 2016 which should improve dedupe performance, also fulls can be splitted from incrementals using veeam v9, so the next months to come will be interesting