Veeam and ReFS deduplication

Availability for the Always-On Enterprise

Veeam and ReFS deduplication

Veeam Logoby olavl » Tue Mar 20, 2018 2:46 pm

Hi,

Veeam has a blogpost regarding using Windows builtin deduplication together with Veeam on a storage server: https://www.veeam.com/blog/data-dedupli ... veeam.html

I assume this is for NTFS deduplication. With Windows Server 2016 build 1709, deduplication was added to ReFS: https://docs.microsoft.com/en-us/window ... erver-1709
Is this a supported, and/or recommended, configuration with Veeam? Does the same configuration apply to ReFS, ref the Veeam blogpost?

regards
Olav Langeland
olavl
Lurker
 
Posts: 1
Liked: never
Joined: Tue Jan 23, 2018 8:21 am
Full Name: Olav Langeland

Re: Veeam and ReFS deduplication

Veeam Logoby SBarrett847 » Tue Mar 20, 2018 4:03 pm

Would be interested to see how well this might work, in addition to block cloning.
SBarrett847
Service Provider
 
Posts: 223
Liked: 34 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Veeam and ReFS deduplication

Veeam Logoby foggy » Mon Apr 02, 2018 1:24 pm

Starting Update 3, Veeam B&R disables block cloning on ReFS volumes with built-in deduplication enabled (unless the repository is CIFS, where we cannot check). The two features cannot be used simultaneously.
foggy
Veeam Software
 
Posts: 16019
Liked: 1227 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: Veeam and ReFS deduplication

Veeam Logoby frankive » Mon Apr 16, 2018 6:37 pm

so what to choose? Refs and block cloning or refs with data deduplication?
frankive
Service Provider
 
Posts: 827
Liked: 98 times
Joined: Tue May 14, 2013 8:35 pm
Location: Norway
Full Name: Frank Iversen

Re: Veeam and ReFS deduplication

Veeam Logoby chjones » Mon Apr 16, 2018 8:43 pm

The best answer is, it depends. And I know that sounds like a copout, but it really does.

If you perform Active Full Backups then stick with deduplication as that will catch duplicate blocks across all files on the volume. Remember, Veeam dedupe only occurs within a single backup file, such as data within a single VBK and not across multiple backup files for the same job. ReFS won't give much, if any, benefit when performing Active Fulls.

However, if you are performing Synthetic Fulls, then ReFS is a good candidate as it provides much faster creation of the synthetic full VBK file. Be sure to check the Veeam B&R console logs to confirm the job is using Fast Clone to perform the synthetic operation. This indicates that Veeam detected the backup repository is ReFS and is leveraging it for your synthetic full creation, which dramatically reduces the time to create the new VBK.

If you saw this week's Veeam Digest you will see that Microsoft have approved ReFS for use on SAN volumes (iSCSi/FC/FCoE) as long as the devices in the path, including the storage array are in the Windows Server Catalog (https://www.windowsservercatalog.com/). Previously Microsoft would only support ReFS when using on direct-attached storage. To quote Gostev in the email, "ReFS is in fact fully supported on ANY storage hardware that is listed on Microsoft HCL". You can read about ReFS here: https://docs.microsoft.com/en-us/window ... s-overview

I work for NetApp so I'm happy to see that all of our current systems are fully supported by Microsoft with Windows Server 2016 so we can use ReFS for Backup Repositories and then use ONTAP's dedupe to get the best of both worlds :D ... although if you were using ONTAP you would be best to leverage ONTAP snapshots and SnapMirror for complete protection whilst still retaining Veeam's features like application-aware processing and SureBackups. But it's great to have the options now.
chjones
Enthusiast
 
Posts: 97
Liked: 26 times
Joined: Tue Oct 30, 2012 7:53 pm
Full Name: Chris Jones

Re: Veeam and ReFS deduplication

Veeam Logoby nmdange » Mon Apr 16, 2018 9:03 pm

I think I've posted this before, but I'm experimenting with this methodology to combine ReFS Fast Clone with Deduplication:
https://charbelnemnom.com/2016/10/how-t ... p-storage/

Essentially, the Veeam backup repository is a VM running on a Hyper-V host. Inside the VM you run ReFS with Fast Clone, and on the host you run NTFS with deduplication (or eventually ReFS with deduplication). I was holding off on this until the ReFS issues have been sorted out, but with the latest updates looking promising, I'm going to start working on moving my repositories into new 2016 VMs and see how it works.
nmdange
Expert
 
Posts: 294
Liked: 69 times
Joined: Thu Aug 20, 2015 9:30 pm

Re: Veeam and ReFS deduplication

Veeam Logoby SBarrett847 » Tue Apr 17, 2018 7:06 am

nmdange wrote:Essentially, the Veeam backup repository is a VM running on a Hyper-V host. Inside the VM you run ReFS with Fast Clone, and on the host you run NTFS with deduplication (or eventually ReFS with deduplication)

I'd be very interested in the IOPs this results in. I've Deduped CSV volumes in the past with much success. However there is a corresponding increase in read IOPs
SBarrett847
Service Provider
 
Posts: 223
Liked: 34 times
Joined: Tue Feb 02, 2016 5:02 pm
Full Name: Stephen Barrett

Re: Veeam and ReFS deduplication

Veeam Logoby frankive » Tue Apr 24, 2018 11:06 am

any suggestions from Veeam on this? refs with block cloning vs refs with deduplication?
frankive
Service Provider
 
Posts: 827
Liked: 98 times
Joined: Tue May 14, 2013 8:35 pm
Location: Norway
Full Name: Frank Iversen


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Google Feedfetcher and 41 guests