Question re ReFS 3 speed benefits and RAID6

Availability for the Always-On Enterprise

Question re ReFS 3 speed benefits and RAID6

Veeam Logoby DaveWatkins » Wed Oct 19, 2016 7:58 pm

This is probably not something anyone can answer for sure at this point, but i'm wondering if some of the Veeam guys or guys with preview access to 9.5 might have a stab at it.

Say we have a repository setup with, say, 20 x 4TB disks currently in RAID10 giving us a 40TB volume and use reverse incremental. Would the block improvements in ReFS 3/Server 2016 make it viable to switch to RAID6 without sacrificing too much overall speed?

Obviously it's a fairly difficult thing to quantify as everyone's specific use case is different but it seems like the sort of thing that a lot of people might be very interested in and I'm wondering if Veeam has done any internal testing which might be available.
DaveWatkins
Expert
 
Posts: 252
Liked: 61 times
Joined: Sun Dec 13, 2015 11:33 pm

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby tsightler » Wed Oct 19, 2016 8:39 pm 2 people like this post

I don't know if anyone has done any testing with reverse incremental since, as a general rule, the forward incremental modes are recommended in the vast majority of cases. That's certainly been the focus of all of my testing to this point.

Since the primary reason to go with RAID10 over RAID6 was due to the IOPS required for synthetic operations I can certainly say that there is almost zero reason to use RAID10 for ReFS with any of the forward incremental options. I suspect it would have a huge impact on performance with reverse incremental as well, but I certainly haven't tested it. I'll set up a test in my lab but it will probably take a day or two to get any reasonable results.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby DaveWatkins » Wed Oct 19, 2016 8:45 pm

Hi Tom

That would be awesome, in our case we went with reverse as we wanted to get full's to tape and doing virtual full's had the same effect as the synthetic operations to the point the array couldn't keep up with the tape drives so we switched over to reverse. Since I'm going to have to reformat anyway to get ReFS it's a perfect time to rethink the RAID level too.

Dave
DaveWatkins
Expert
 
Posts: 252
Liked: 61 times
Joined: Sun Dec 13, 2015 11:33 pm

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby tsightler » Thu Oct 27, 2016 3:45 am 12 people like this post

So I have done some basic testing, enough to say that reverse incremental with ReFS+block clone is similar in performance to forward incremental, although the exact profile is slightly different. My test is not on a huge dataset, but it is directed at a "worst case" storage system, a RAID5 array with only three 3TB SATA drives, so not much performance there.

The backup size source data size is ~435GB, with about 300GB used, and the full backup is roughly 120GB on disk after compression. These test were done with per-VM backup files so keep that in mind. Typical daily change was around 100GB raw, 25-30GB in the VIB/VRB file after compression/dedupe. The exact same VMs were backed up to four different repository type using the same back end disks:

NTFS -- Forward Incremental
NTFS -- Reverse Incremental
ReFS -- Forward Incremental
ReFS -- Reverse Incremental

Full backups performance was, as expected, very similar for all cases, with ReFS being about 3-5% slower, but I'd consider the difference non-interesting and would really need to run the test multiple times to make sure it wasn't just minor testing variation. It was incremental backups that we were interested in, below I'll share the averages of these jobs over 7 days, as well as the worst case times, to give some idea of the performance difference. For forward incremental I'll break out the backup time vs the merge time because I think that's important in indicating the differences in snapshot open times for VMs vs disk processing time.

All Jobs
Average Data Read/day: ~90GB
Average backup size/day: ~30GB
Worst case was a change rate of nearly 135GB

NTFS -- Forward Incremental
Average time to backup all 7 VMs/day: ~11min (worst case was ~20 min)
Average merge time/day: ~24min (worse case was ~44 min)
Average total time/day: ~35min

NTFS -- Reverse Incremental
Average time to backup all 7 VMs/day: ~45min (worst case was ~70 min)

ReFS -- Forward Incremental
Average time to backup all 7 VMs/day: ~11min (worst case was ~20 min)
Average merge time/day: ~2m: (worse case was ~3 min)
Average total time/day: ~13min

ReFS -- Reverse Incremental
Average time to backup all 7 VMs/day: ~13min (worst case was ~20 min)

As you can see, in this fairly small scale testing, reverse incremental was roughly 3.5x faster on ReFS vs NTFS on the same back end disk storage, that's a pretty huge improvement and should be more than enough to make up for the switch from RAID10 to RAID6, especially since the sequential write performance should actually increase with RAID6. The move to ReFS + block clone so significantly reduces the random I/O requirement that the major benefit of moving from RAID6 to RAID10, which was increased random I/O, doesn't really exist anymore.

I'll try to stand up and perform a more scale test in the future, this was really more of an informal test since I had a few spare cycles in the lab, but I think it indicates the huge benefits of ReFS + block clone even for reverse incremental.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby DaveWatkins » Thu Oct 27, 2016 3:49 am

Wow, that's much better than I was expecting and i had pretty high hopes.

Thanks for this, it's awesome and makes the transition to RAID6 a non-issue.

Dave
DaveWatkins
Expert
 
Posts: 252
Liked: 61 times
Joined: Sun Dec 13, 2015 11:33 pm

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby HannesK » Tue Nov 08, 2016 2:56 pm 3 people like this post

I also did some tests that might be relevant for really small environments or branch offices. It shows that even in small environments the benefit of ReFS is significant.

My setup:
Image
Results for Backup:
Image

I has also some positive effects on disk usage:
Image

wheras the filesizes shown in explorer are different:
Image

Restore speed for full disk restore and ReFS restore are almost equal in my environment because I only have 1GBit network...
HannesK
Veeam Software
 
Posts: 1144
Liked: 144 times
Joined: Mon Sep 01, 2014 11:46 am

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby lando_uk » Tue Nov 08, 2016 3:49 pm

This is pretty exciting. RAID10 performance using RAID6.

I presume Restores will be the same, or does it manage large fragmented files better?

Thanks.
lando_uk
Expert
 
Posts: 236
Liked: 17 times
Joined: Thu Oct 17, 2013 10:02 am
Full Name: Mark

Re: Question re ReFS 3 speed benefits and RAID6

Veeam Logoby jronnblom » Thu Dec 08, 2016 11:05 am

Have you at Veeam done any more testing?

Or does someone have experience from running it in production?

-J
jronnblom
Service Provider
 
Posts: 15
Liked: 1 time
Joined: Wed Oct 23, 2013 6:15 am
Full Name: Janåke Rönnblom

[MERGED] Reverse incremental backups to Refs???

Veeam Logoby radkins22 » Tue Dec 27, 2016 10:11 pm

On another thread (linked below) I saw where tsightler had done some basic testing where reverse incrementals were about 3.5 times faster when going to Refs when compared to NTFS. I have done similar testing and I only see about a 5% increase in speed. When I do forward incrementals the difference in speed is night and day, but when doing reverse incrementals not so much. Just wondering what kind of performance others are seeing when doing reverse incrementals to ReFS repositories.

veeam-backup-replication-f2/question-re-refs-3-speed-benefits-and-raid6-t38331.html
radkins22
Lurker
 
Posts: 2
Liked: never
Joined: Tue Nov 08, 2016 2:20 pm
Full Name: Ronald Adkins


Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Google Feedfetcher, mwpreston, rkovhaev, Yahoo [Bot] and 44 guests