Discussions specific to the VMware vSphere hypervisor
Post Reply
mikegodwin
Enthusiast
Posts: 53
Liked: 1 time
Joined: Oct 12, 2012 12:28 am
Full Name: Mike Godwin
Contact:

Backup much faster to ReFS than NTFS?

Post by mikegodwin » Aug 29, 2017 2:48 pm

Is there any reason backups would be slower to an NTFS volume instead of a ReFS volume? I run a test backup job and it runs significantly faster to the ReFS volume. Same job settings, same VM getting backed up, both volumes have 64K allocation unit size. Both repository volumes are on the same target server. The ReFS job always runs between 500-800 MB/sec and I can run a backup every hour (job takes 10 minutes), while the NTFS job typically runs at around 50-60 MB/sec and runs all day long and never completes. I check the backup target server and it looks like it's doing nothing while the NTFS job is running (CPU almost zero, 50% RAM usage of 16GB, almost zero disk I/O). What else can I check to try and speed up the NTFS job? ReFS runs great when it works but after a while it starts to have problems (server locks up, etc), so I don't see ReFS as an option and need to get NTFS working at a better speed.

foggy
Veeam Software
Posts: 17089
Liked: 1396 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by foggy » Aug 29, 2017 4:03 pm

Hi Mike, ReFS is faster due to the fastclone functionality. What is the bottleneck for the NTFS job?

mikegodwin
Enthusiast
Posts: 53
Liked: 1 time
Joined: Oct 12, 2012 12:28 am
Full Name: Mike Godwin
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by mikegodwin » Aug 29, 2017 6:00 pm

This difference exists even when not running synthetic fulls, just regular incrementals. So I think fast clone is not used in those cases, is that correct? With the ReFS job the bottleneck is the source (even though it's all flash source) and the NTFS job bottleneck is the target.

mikegodwin
Enthusiast
Posts: 53
Liked: 1 time
Joined: Oct 12, 2012 12:28 am
Full Name: Mike Godwin
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by mikegodwin » Aug 30, 2017 5:08 pm

I also noticed that one job with "Enable application-aware processing" disabled going to an ReFS repository runs extremely fast. The processing rate shows 2GB/sec. Will have to test more to see if there is a difference with application-aware processing enabled vs disabled. It seems like ReFS is doing something with caching file system info in memory to make the jobs run faster, and it also seems like that's the source of some of the ReFS stability problems discussed in other threads.

foggy
Veeam Software
Posts: 17089
Liked: 1396 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by foggy » Sep 01, 2017 3:08 pm

mikegodwin wrote:This difference exists even when not running synthetic fulls, just regular incrementals. So I think fast clone is not used in those cases, is that correct?
Fastclone is used during merge of increments as well, so if you're using forever forward incremental backup mode, there should be a benefit.
mikegodwin wrote:I also noticed that one job with "Enable application-aware processing" disabled going to an ReFS repository runs extremely fast. The processing rate shows 2GB/sec. Will have to test more to see if there is a difference with application-aware processing enabled vs disabled.
Application-aware processing is a source thing, so is not impacted by the repository type.

jmmarton
Veeam Software
Posts: 1284
Liked: 145 times
Joined: Nov 17, 2015 2:38 am
Full Name: Joe Marton
Location: Chicago, IL
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by jmmarton » Sep 14, 2017 7:27 pm

Fastclone is used during the merge/transform and shouldn't have any impact on reading the source data. I wonder if there are other job differences between the two jobs considering the huge difference in throughput and difference in bottleneck as well. What sort of underlying disk is in use on the repo and does it differ from the ReFS volume vs NTFS volume?

Joe

dellock6
Veeam Software
Posts: 5542
Liked: 1535 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Backup much faster to ReFS than NTFS?

Post by dellock6 » Sep 24, 2017 4:59 pm

Actually, if the underlying server is properly sized (read: it's not lacking RAM, as this is the main problem with ReFS), ReFS should be, in general, faster than NTFS with Integrity Streams enabled, which is what Veeam enables by default on any repository. This is mainly caused by the log-structured filesystem in ReFS, there's a nice article here if you want to learn more: https://www.starwindsoftware.com/blog/refs-performance
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com/en/
vExpert 2011-2012-2013-2014-2015-2016-2017-2018
Veeam VMCE #1

Post Reply

Who is online

Users browsing this forum: Exabot [Bot], ViciousXUSMC and 46 guests