Dedupe and REFS on the way?

[mongie]

Hello folks...

Was just reading through the release notes for Windows Server Insider Preview Build 16257 and I noticed a new feature listed that caught my eye...

"Data Deduplication available for ReFS"

Is that something that Veeam is going to support (assuming it makes it to Server 2016 proper)

https://blogs.windows.com/windowsexperi ... ild-16257/

[mkretzer]

From the veeam digest from 4 weeks ago:

Of course, as it comes to Veeam, the most interesting announcement is about Windows Server deduplication coming to ReFS! There's also some "ReFS compaction" feature, which obviously can't be bad by the sound of it – but I've no idea what it is about, and trying to find out. Anyhow, in the last few days, many people pinged me to share their excitement about ReFS deduplication specifically, but unfortunately I had to cool them down a bit.

First, I understand that this deduplication will not be block clone aware: performing a block clone operation that involves deduped data will require the cloned block to be rehydrated first – which will obviously kill all performance benefits of fast cloning. So I would not recommend using ReFS dedupe for primary and secondary backup repositories with Veeam for sure – however, there's one use case where it will shine, and that is the Archive Tier functionality of Veeam Backup & Replication v10. Indeed, ReFS Storage Spaces with dedupe enabled makes the perfect long-term archival solution: you still get proper variable block size deduplication as today with NTFS, but you also get the reliability of ReFS with its data integrity streams and automated data corruption self-healing. Except one little problem...

Because Microsoft is reusing the existing, proven NTFS deduplication engine, we should expect the same max volume size of 64TB also with ReFS – which really limits the applicability of this technology as it comes to backup repositories to smaller shops. Oh well, maybe I have no rights to be upset, because who said Microsoft wants to go competing head to head with deduplication storage vendors anyway? Although to me, the bigger issue is that they are seemingly building lots of new functionality around ReFS instead of focusing on making what is already generally available more stable. I just don't get that – for Veeam, resolving support issues is always #1 priority for R&D. Yes, often this means some new features postponed, which always hurts me as the Product Manager – but I'm still the firm believer it is the only right thing to do.