Comprehensive data protection for all workloads
Post Reply
b.vanhaastrecht
Service Provider
Posts: 833
Liked: 154 times
Joined: Aug 26, 2013 7:46 am
Full Name: Bastiaan van Haastrecht
Location: The Netherlands
Contact:

ReFS and slow file copy speed

Post by b.vanhaastrecht »

Hello,

We use SOBR with 4 extends, all are 10TB in size and are formatted ReFS 64k on one single Windows 2016 box. Those extends are backed by seperate VMWare datastores on NetApp NFS volumes.

We recently needed to manually move/copy some .vib and .vbs files cross volume/extend. The file copy was initiated within the 2016 box from volume to volume. We notice the copy speed was not going faster than about 50MB/s. We started to investigate as we would expect much higher speeds.

We noticed no difference in .vbs or .vib file, but the file age does make a difference. Files older than one day were stuck at 50MB/s and fresh files were going 200MB/s. The backed NetApp does use post compression and deduplication, but with tests on a seperate machine the deduped and compressed files were actually faster in copy then non toughed files, so it ruled out the NetApp. There's no dedupe enabled in Windows, so we can't find a cause why +1 day files copy only at 50MB/s.

Anyone has any ideas about this?

Regards,
Bastiaan
======================================================
Veeam ProPartner, Service Provider and a proud Veeam Legend
b.vanhaastrecht
Service Provider
Posts: 833
Liked: 154 times
Joined: Aug 26, 2013 7:46 am
Full Name: Bastiaan van Haastrecht
Location: The Netherlands
Contact:

Re: ReFS and slow file copy speed

Post by b.vanhaastrecht »

I would like to add: When copying non-veeam files the speed is always as expected: 200MB/s. So only Veeam .vbs and .vib files are affected.
======================================================
Veeam ProPartner, Service Provider and a proud Veeam Legend
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: ReFS and slow file copy speed

Post by Gostev »

Fragmentation due to block cloning?
b.vanhaastrecht
Service Provider
Posts: 833
Liked: 154 times
Joined: Aug 26, 2013 7:46 am
Full Name: Bastiaan van Haastrecht
Location: The Netherlands
Contact:

Re: ReFS and slow file copy speed

Post by b.vanhaastrecht »

Makes sense with .vbk's, but we are seeing the same with .vib's. Those arent formed via block cloning, right?
======================================================
Veeam ProPartner, Service Provider and a proud Veeam Legend
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: ReFS and slow file copy speed

Post by Gostev »

You are correct, however future VBKs will reference blocks brought by earlier VIBs, which may in turn affect VIB blocks metadata and thus slow down VIBs processing. ReFS architecture is not a public knowledge, so I am just theorizing - obviously, this question is better to be addressed to Microsoft Support.
pkuikman
Service Provider
Posts: 53
Liked: 7 times
Joined: May 04, 2018 1:54 pm
Full Name: Peter Kuikman
Contact:

Re: ReFS and slow file copy speed

Post by pkuikman »

b.vanhaastrecht wrote: Aug 08, 2018 11:18 am Hello,

We use SOBR with 4 extends, all are 10TB in size and are formatted ReFS 64k on one single Windows 2016 box. Those extends are backed by seperate VMWare datastores on NetApp NFS volumes.

We recently needed to manually move/copy some .vib and .vbs files cross volume/extend. The file copy was initiated within the 2016 box from volume to volume. We notice the copy speed was not going faster than about 50MB/s. We started to investigate as we would expect much higher speeds.

We noticed no difference in .vbs or .vib file, but the file age does make a difference. Files older than one day were stuck at 50MB/s and fresh files were going 200MB/s. The backed NetApp does use post compression and deduplication, but with tests on a seperate machine the deduped and compressed files were actually faster in copy then non toughed files, so it ruled out the NetApp. There's no dedupe enabled in Windows, so we can't find a cause why +1 day files copy only at 50MB/s.

Anyone has any ideas about this?

Regards,
Bastiaan
It's an old topic, but did you ever figure out how to speed up the copy/paste action?
b.vanhaastrecht
Service Provider
Posts: 833
Liked: 154 times
Joined: Aug 26, 2013 7:46 am
Full Name: Bastiaan van Haastrecht
Location: The Netherlands
Contact:

Re: ReFS and slow file copy speed

Post by b.vanhaastrecht » 1 person likes this post

Nope, just a workaround. Use robocopy with /MT:16 or higher to utilize multiple threads while copying files.
======================================================
Veeam ProPartner, Service Provider and a proud Veeam Legend
Post Reply

Who is online

Users browsing this forum: No registered users and 164 guests