Comprehensive data protection for all workloads
Gostev
SVP, Product Management
Posts: 24092
Liked: 3280 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: ReFS 3.0 and Dedup

Post by Gostev » Feb 16, 2019 2:20 am 1 person likes this post

Hi Mark, this was the last minute development and we did not have any chance to perform effectiveness testing (only reliability testing), which is why the feature is declared as experimental. Also, the idea was to hear the effectiveness results from the community, as lab environment don't have real-world data anyway. Thanks!

DonZoomik
Expert
Posts: 101
Liked: 25 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by DonZoomik » Feb 19, 2019 7:46 am 1 person likes this post

Case resolved by support.
Slowness has be reproduced in lab and it's as slow as with NTFS deduplication.

absentminded
Lurker
Posts: 2
Liked: never
Joined: Mar 13, 2019 12:11 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by absentminded » Mar 13, 2019 12:14 pm

Our fast clone synthetic takes hours. How was your case resolved?

DonZoomik
Expert
Posts: 101
Liked: 25 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by DonZoomik » Mar 13, 2019 1:38 pm

It's wasn't resolved per se, just replicated in lab and confirmed to work... badly.
Summary is that all block clone benefits are lost if file is deduplicated. ReFS deduplication has no real integration with block clone and metadata operations become full read and write operations (copied from dedup store to new blocks). Also, written data is no longer deduplicated.
TL;DR: fast clone is slow with ReFS Deduplication.

jorgedlcruz
Veeam Software
Posts: 237
Liked: 103 times
Joined: Jul 17, 2015 6:54 pm
Full Name: Jorge de la Cruz
Contact:

Re: ReFS 3.0 and Dedup

Post by jorgedlcruz » Mar 13, 2019 1:50 pm 1 person likes this post

Hi guys,
But that will be expected I will say isn't? To obtain the best possible performance, at a logical level really, you will need to use the usual ReFS for your synthetic, might be weekly or so, and then once you know your synthetic file is created, that is the file you want to apply dedupe.

So, look at your Windows Deduplication schedule and make sure you are not doing dedupe on the files of the open chain which will be used to create the synthetic, as if you do the server will need to dedupe all that files before do the ReFS, which will end in that long time to do the fast-clone.

I see this ReFS 3.0 and Dedupe kind of what Exagrid does, you have a landing zone for your weekly, where you run your synthetic operations, and then after you apply your dedupication to the chains which are closed already. Of course all of this is DIY vs. ExaGrid which gives you all of this and more out of the box.
Jorge de la Cruz
Systems Engineer UKI @ Veeam Software

@jorgedlcruz
https://www.jorgedelacruz.es / https://jorgedelacruz.uk
vExpert 2014-2019 / Nutanix Technology Champion 2018-2019

absentminded
Lurker
Posts: 2
Liked: never
Joined: Mar 13, 2019 12:11 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by absentminded » Mar 13, 2019 2:23 pm

I confirm we do dedup files older than one day. Have to try eight days.

frankive
Service Provider
Posts: 878
Liked: 102 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: ReFS 3.0 and Dedup

Post by frankive » May 13, 2019 11:02 am

soo... I should stick with server 2019 and Refs and not enabling deduplication? :)

m.novelli
Veeam ProPartner
Posts: 353
Liked: 40 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: ReFS 3.0 and Dedup

Post by m.novelli » May 13, 2019 2:47 pm

Hi guys, slight off topic... does Veeam support restore of deduplicated files stored on a source VM Windows Server 2019 , ReFS 3.0 volume?

Do I need to have Veeam installed also on box with Windows Server 2019 with "Data Deduplication" feature installed?

Thanks,

Marco

DonZoomik
Expert
Posts: 101
Liked: 25 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by DonZoomik » May 13, 2019 2:55 pm

Mount server must have Windows Server 2019 and data deduplication feature installed.

DonZoomik
Expert
Posts: 101
Liked: 25 times
Joined: Nov 25, 2016 1:56 pm
Contact:

Re: ReFS 3.0 and Dedup

Post by DonZoomik » May 13, 2019 2:58 pm

frankive wrote:
May 13, 2019 11:02 am
soo... I should stick with server 2019 and Refs and not enabling deduplication? :)
While I was running it, merge seemed a bit more stable than with NTFS deduplication - no other benefits. With NTFS, I got occasional transient errors during merge (probably due to file locking from deduplication engine).

csydas
Expert
Posts: 176
Liked: 44 times
Joined: Jan 16, 2018 5:14 pm
Full Name: Harvey Carel
Contact:

Re: ReFS 3.0 and Dedup

Post by csydas » May 23, 2019 6:05 pm

Hi Veeam,

Does this key work if the Windows 2019 ReFS Volume is presented as CIFS? Or does Veeam need to have the volume added as a Windows Server?

HannesK
Veeam Software
Posts: 3062
Liked: 371 times
Joined: Sep 01, 2014 11:46 am
Location: Austria
Contact:

Re: ReFS 3.0 and Dedup

Post by HannesK » May 24, 2019 6:21 am

Hello,
yes it does fastclone via SMB / CIFS on a REFS dedupe filesystem. I just tested it in my lab.

Anyway, I'm not sure whether it makes any sense. SMB is the slowest way for backups and the comments say that ReFS dedupe & fastclone is also slow :-)

Best regards,
Hannes

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 15 guests